How to design an RPC system

This article is published by the Cloud + community

RPC is a convenient programming model for network communication. Due to its high combination with programming language, it greatly reduces the complexity of network data processing and improves the readability of code. However, the composition of RPC itself is quite complex, and there are a lot of compromises and trade-offs due to the constraints of programming language, network model and usage habits. This article is through the analysis of several popular RPC implementation cases, to provide you in the design of RPC system reference.

Because the network development of RPC bottom layer is generally related to the specific use environment, and the means of programming implementation is also very diversified, but does not affect users, so this paper basically involves how to implement an RPC system.

Understanding RPC (Remote Calls)

We’ve all come across the concept of “remote invocation” in various operating systems and programming language ecosystems. Generally speaking, they refer to a simple line of code that calls a program on another computer over the network. Such as:

• RMI — Remote Method Invoke: Invoke a Remote Method. Methods are usually attached to an object, so RMI usually refers to a call to a method function on an object on a remote computer.
• RPC — Remote Procedure Call: Remote Procedure Call This is a call to a particular piece of function code on another computer on the network.

Remote call itself is a concept of network communication, which is characterized by encapsulating network communication into a function like call. In addition to remote call, network communication generally has several other concepts: packet processing, message queue, flow filtering, resource pull waiting. Here’s a comparison of the differences:

For the characteristics of remote call – call a function. Similar solutions have been developed in various languages, while some have attempted to cross languages. While remote calls may seem the easiest to use programmatically, they also have significant disadvantages. So understanding the advantages and disadvantages of remote invocation is a key issue in deciding whether to develop or use the remote invocation model.

The advantages of remote invocation are:

• The network layer is masked. Therefore, in the transmission protocol and coding protocol, we can choose different schemes. For example, WebService scheme uses HTTP transport protocol +SOAP encoding protocol; REST solutions tend to use the HTTP+JSON protocol. Facebook Thrift can even customize any different transport Protocol and encoding Protocol, you can use TCP+Google Protocol Buffer, UDP+JSON… . Because the network layer is shielded, you can optimize the network part independently of the actual needs, without the need for processing code involving business logic, which is very valuable for applications that need to run in a variety of network environments.
• Function mapping protocol. Instead of writing a lot of coding protocols and subcontracting logic, you can write data structures and function definitions directly in a programming language. For systems with very complex business logic, such as online games, you can save a lot of time defining message formats. And the function call model is very easy to learn, no need to learn communication protocols and processes, making it easy for inexperienced programmers to start using network programming.

Disadvantages of remote invocation:

• Increased performance cost. Because network traffic is packaged as a “function,” a lot of extra processing is required. For example, you need to pre-produce code or use reflection. These are all operations that consume additional CPU and memory. Moreover, in order to express complex data types, such as variable length string/map/list, more descriptive information needs to be added to the packet, which will occupy more network packet length.
• Unnecessary complication. If you want to use the HTTP/FTP protocol model only for specific business needs, such as transferring a fixed file. If it is for monitoring or IM software, it is faster and more efficient to send and receive messages with simple message encoding. If you want to be a proxy server, streaming is easy. Also, if you want to do data broadcasting, message queues are easy to do, while remote calls are almost impossible to do.

Therefore, remote invocation is most suitable for scenarios where service requirements and network environment are changeable.

The core problem of RPC scheme

Since the interface used for remote calls is a “function,” there are three areas where decisions need to be made about how to construct this “function” :

1. How to express “remote” information

Remote means another location on the network, so the network address is the part that must be entered. On TCP/IP networks, the IP address and port number represent an entry point to a running program. So specifying the IP address and port is required to make the remote call.

However, a program may run many functions and receive multiple remote calls with different meanings. How to get the user to specify the different meanings of the remote call entry becomes another issue. Of course, the simplest is one call per port, but an IP supports up to 65535 ports, and other network functions may also need ports, so this scheme may not be enough, and a number represents a function is not very good to understand, you must look up the table to understand.

So we have to think of something else. Under the idea of object oriented, some schemes are proposed: to summarize different function combinations by different objects, specify objects first, and then specify methods. This idea is very much in line with the way programmers understand it, and EJB is the solution. Once you have identified the object model for defining the address of the remote call, you need a way to specify the remote object. In order to specify the object, you must be able to transmit some information about the object from the called (server side) to the caller (client side).

The simplest solution is for the client to enter a string as the “name” of the object, send it to the server, find the object registered with the “name”, if found, the server will use some technique to “transfer” the object to the client, and then the client can call its method. Of course, such transmission is not to copy the object data on the whole server to the client, but to use some symbols or signs to represent the object on the server, and then sent to the client.

If you are not using an object-oriented model, a remote function must be located and transported, because the function you call must be found and then become an interface on the client side before it can be called. How to express “remote objects” (including object-oriented objects or just functions) to be located on the network; The first important question in the “remote call” design scenario is how to make it available to the client once the location is successful.

2. How should the interface form of a function be expressed

Remote invocation, constrained by network communication, often does not fully support all the features of a programming language. Pointer type parameters in C functions, for example, cannot be passed over the network. Therefore, the definition of functions for remote calls, and what features of the language can and cannot be used, need to be specified in the design scheme.

If this rule is too strict, it will affect the user’s ease of use; If it is too broad, it can lead to poor performance of remote calls. How to design a way to describe a function in a programming language as a function that is called remotely is also an issue to consider. Many take the generic approach of configuration files, while others add special comments directly to the source code.

In general, a compiled language such as C/C + + source code is the only method to be adopted according to the configuration file generated by the scheme, a virtual machine language such as C # / JAVA reflection mechanism can be used in combination with configuration files (set in the source code with special notes instead of a configuration file), if it is a scripting language is more simple, sometimes don’t even need a configuration file, Because the script can do that on its own. In short, it is also a matter of careful consideration what constraints the interface for remote calls satisfies.

3. What method is used to realize network communication

The most important implementation detail of remote invocation is about network communication. The question of which mode of communication is used to host remote calls is boiled down to two sub-questions: what server provides network functionality? What kind of communication protocol?

The remote calling system can directly program TCP/IP itself to achieve communication, or it can entrust some other software, such as Web server, message queue server and so on… You can also use different network communication frameworks, such as Netty/Mina open source frameworks. Communication protocols generally have two layers: one is the transport protocol, such as TCP/UDP or a higher level HTTP, or their own defined transport protocol; The other is encoding Protocol, which is how to serialize and deserialize objects in a programming language into binary byte streams. Popular solutions include JSON, Google Protocol Buffer, etc. Many development languages also have their own serialization schemes, such as JAVA/C#. These technical details, which you choose to use, have a direct bearing on the performance and environmental compatibility of the remote calling system.

The above three problems are the core selection that the remote call system must consider. Depending on the constraints each solution faces, they make trade-offs on these three issues to adapt to their constraints. But there is no “one-size-fits-all” or “one-size-fits-all” solution for this reason: in such a complex system, the more features you have to take care of, the more costs (ease of use, performance overhead) you have to pay.

Let’s take a look at the various remote invocation solutions that exist in the industry and see how they balance and choose between these three areas.

Examples of Industry Solutions

1. CORBA

CORBA is an “old” and ambitious scheme, which tries to accomplish the task of communication across languages as well as remote calls. Therefore, it is the most complex, but its design ideas have been studied by many other schemes later. In the location of communication object, it uses URL to define a remote object, which is very easy to accept in the Internet age. The contents of its objects are limited to C language types and can only pass values, which is also easy to understand. In order to allow programs of different languages to communicate, it is necessary to independently design a Language for describing remote interfaces, which is called IDL: Interface Description Language.

With this approach, you can define your interface in a language that stands out from all other languages, and then use tools to automatically generate code for various programming languages. This is almost the only option for compiled languages. CORBA does not make any conventions about communication, leaving it to implementers of specific languages, which may be one reason why it has not gained widespread popularity.

CORBA actually has a very famous successor, the Thrift framework for Facebook inc. Thrift is also a remote call scheme that uses IDL compilation to generate multiple languages and fully implements communication bearing in C++/JAVA and other languages, so it is a particularly attractive one in open source frameworks. Another feature of Thrfit’s communication bearer is that it can combine various transport and encoding protocols, such as TCP/UDP/HTTP with JSON/BIN/PB… This allows it to choose almost any network environment.

The Thrift model is similar to the following figure. The stub stands for stub code, which is a functional program that the client uses directly. Skeleton stands for “skeleton code” and requires programmers to write template code that provides specific remote service functionality, usually by filling in blanks or inheriting (extending) the template. This stub-skeleton model is standard for almost all remote invocation schemes.

2. JAVA RMI

JAVA RMI is a remote invocation scheme that comes with the JAVA virtual machine. It can also locate remote objects using urls and pass parameter values using JAVA’s native serialization encoding protocol. In terms of Interface description, since this is a Java-only scenario, the JAVA language Interface type is used directly as the definition language. The user implements this interface type to provide a remote service, and JAVA automatically generates client-side calling code from the interface file for the caller to use. His underlying communication, or TCP protocol to achieve. In this case, the Interface file is the JAVA language IDL and skeleton template for developers to fill in remote service content. Stub code is handled directly by the VIRTUAL machine due to JAVA reflection.

This solution is very simple to use because of the JAVA virtual machine support. It is easy to solve the problem by completely following the JAVA programming method of the logo, but it can only run in the JAVA environment, which limits its scope of application. You can’t have your cake and eat it, usability and usability are often in conflict. This is very different from CORBA/Thrift’s pursuit of maximum applicability, which leads to the difference in ease of use.

3. Windows RPC

RPC support in Windows is early and well-established. It first queries objects through GUids, then uses C language types as parameter values. Since the Windows API is mainly C, for RPC functionality, an IDL is used to describe the interface, and finally.h and.c files are generated to produce RPC stub and skeleton code. And the communication mechanism, because the operating system comes with it, so it uses the kernel LPC mechanism to carry it, which is more convenient for users. But it is also limited to making calls between Windows programs.

4. WebService & REST

In the Internet age, programs need to call each other over the Internet. The most popular protocols on the Internet are HTTP protocol and WWW Service, so the Web Service using HTTP protocol naturally becomes the most popular scheme of cross-system call. With access to most of the Internet infrastructure, Web services are almost a no-brainer to develop and implement. In general, it uses urls to locate remote objects, and parameters are passed through a set of predefined types (mainly C-base types), as well as object serialization. For interface generation, you can parse HTTP directly yourself, or you can use a specification such as WSDL or SOAP. In the REST solution, only PUT, GET, DELETE, and POST operation functions are limited, and the other functions are parameters.

Summarizing the RPC scenarios above, we found that the industry generally has several options for addressing the three core issues of remote invocation:

• Remote object location: using urls; Or use a name service to look it up
• Remote call parameter passing: use C’s basic type definition; Or use some sort of scheduled serialization (deserialization) scheme
• Interface definition: a predefined interface definition file, using a specific format technique; Or use some description protocol IDL to generate these interface files
• Communication hosting: there are servers that use specific TCP/UDP servers, and there are communication models that allow users to develop their own custom. There are also more advanced transport protocols such as HTTP or message queues

Scheme selection

After we have identified several feasible options for the remote call system scheme, it is natural to clarify the advantages and disadvantages of each scheme, so as to select a design that really fits the requirements:

**1. Description of remote objects: ** The use of URL is a common standard on the Internet, which is convenient for users to understand and easy to add content that needs to be extended in the future, because the URL itself is a string composed of multiple parts; The name service is older, but it still has its advantages, that is, the name service can be attached with a series of features such as load balancing, disaster recovery capacity expansion, and custom routing, which is easier to implement for complex requirements.

**2. Remote call interface description: ** If only limited to a language, operating system, platform, direct use of “metaphorical” interface description, or “annotation” type annotation means to annotate the source code, remote call interface definition, is the most convenient. However, if you want to be compatible with compiled languages such as C/C++, you must use some KIND of IDL to generate the source code for those compiled languages.

**3. Communication bearer: ** Provides users with their own customized communication modules, which can provide the best applicability, but also allows users to increase the complexity of use. However, HTTP/ message queue is relatively simple in system deployment, operation and maintenance, and programming. The disadvantage is that the space for customization of performance and transmission characteristics is relatively small.

After analyzing the core issues, we also need to consider some applicable scenarios:

**1. Object-oriented or procedural: ** If we’re just thinking about making process-oriented remote calls, we just need to locate “functions”. If it is object-oriented, you need to locate the object. Since functions are stateless, the location process can be as simple as a name, while objects need to dynamically look up their IDS or handles.

** In single-language scenarios, header files or interface definitions are handled entirely in one language. If they are cross-language, IDL is required

** Hybrid hosting or HTTP server hosting: ** Hybrid hosting may use TCP/UDP/ shared memory and other low-level technologies, which can provide optimal performance, but must be very troublesome to use. Using HTTP server, it is very simple, because WWW service open source software, many libraries, and the client can be debugged using a browser or some JS pages, the disadvantage is its low performance.

If we were to design a remote call system for a domain where the business logic is very variable, such as the enterprise business application domain, or the game server-side domain, we might choose one of the following:

**1. Use name services to locate remote objects: ** Since enterprise services require high availability, use name services to identify and select available service objects when querying names. Ejbs (Enterprise Javabeans) in J2EE solutions are served by names.

** Use IDL to generate interface definitions: ** Because enterprise services or game services may not be developed in a uniform language or require a high-performance programming language such as C/C++, only IDL can be used.

* * 3. The use of hybrid communication load: * * although enterprise service looks don’t need to run under the complicated network, but the network environment of different companies may be differ in thousands ways, so want to be a general system, it is better to take the trouble to provide hybrid communication load, it can choose in various protocols such as TCP/UDP.

This article has been published by Tencent Cloud + community authorized by the author

For more fresh technology dry goods, you can pay attention to our Tencent cloud technology community – cloud plus community official number and Zhihu organization number