Micro services are inseparable from RPC framework. The principle, practice and details of RPC framework are the contents to be shared in this article.

What are the benefits of servitization?

One of the benefits of servitization is that it can achieve technology decoupling across teams in large companies without limiting the technology selection used by service providers, as shown in the figure below:

  • Service A: European team maintenance, technical background is Java
  • Service B: maintained by the americas team and implemented in C++
  • Service C: China team maintenance, technology stack is GO

The upstream service caller can invoke the remote service according to the interface and protocol.

But in fact, most Internet companies, with limited r&d teams, mostly use the same technical system to achieve services:

Thus, if there is no unified service framework, the service providers of each team will need to implement one of their ownSerialization, deserialization, network frameworks, connection pooling, sending and receiving threads, timeout handling, state machinesAnd other “out of business” repetitive technical labor, resulting in overall inefficiency.

Therefore, the unified service framework to unify the above “out-of-business” work is the primary problem to be solved by servitization.

What is RPC?

Remote Procedure Call Protocol.

What is “remote” and why “remote”?

Let’s take a look at what is “near”, that is, “local function call”.

When we write:

int result = Add(1, 2);

What’s going on in this line of code?

  • Pass two incoming parameters
  • A function in the local code snippet is called to perform the arithmetic logic
  • Returns an output parameter

All three of these actions, all in the same process space, are local function calls.

Is there a way to call a cross-process function?

Typically, this process is deployed on another server.

The easiest way to think about it is that two processes agree on the same protocol format and use sockets to communicate:

  • The ginseng
  • Which function to call
  • The ginseng

If implemented, this is a “remote” procedure call.

Socket communication can only pass continuous byte streams. How to put input parameters and functions into continuous byte streams?

Suppose we design an 11-byte request message:

  • Fill the first three bytes with the function name “add”
  • Fill the middle 4 bytes with the first parameter “1”.
  • Fill the last four bytes with the second parameter “2”

Similarly, a 4-byte response message can be designed:

  • 4 bytes to fill the result “3”

The caller’s code might become:

Request = MakePacket(” add “, 1, 2);

SendRequest_ToService_B(request);

response = RecieveRespnse_FromService_B();

int result = unMakePacket(respnse);

The four steps are:

(1) Turn the incoming parameter into a byte stream;

(2) send the byte stream to service B;

(3) receive return byte stream from service B;

(4) Change the return byte stream into outgoing parameter;

The code for the server might become:

request = RecieveRequest();

args/function = unMakePacket(request);

result = Add(1, 2);

response = MakePacket(result);

SendResponse(response);

The 5 steps are easy to understand:

(1) The server receives the byte stream;

(2) Turn byte flow into function name and parameter;

(3) Call the function locally to get the result;

(4) Convert the result into a byte stream;

(5) Send the byte stream to the caller;

This process is depicted in a diagram as follows:

The processing steps are clear for both the caller and the server.

What’s the biggest problem with this process?

The caller is too cumbersome to pay attention to many low-level details each time:

  • Serialization of application layer protocol details by converting input parameters to byte streams

  • Socket sending, the details of the network transport protocol

  • The socket to receive

  • Byte stream to outgoing parameter conversion, i.e., deserialize application layer protocol details

Can the calling layer not pay attention to this detail?

Yes, the RPC framework solves this problem by allowing callers to “call remote functions (services) as if they were local functions.”

At this point, do you get a feel for RPC, serialization of the serialization paradigm? Read on for more low-level details.

What does the RPC framework do?

RPC framework to shield complexity from callers and from service providers:

  • The service caller client feels like calling a local function to call the service
  • The service provider Server implements the service as if it were a local function

Therefore, the whole RPC framework is divided into client part and server part. It is the responsibility of RPC framework to achieve the above objectives and shield complexity.

As the picture above shows,Responsibilities of the business sideIs this:

  • Caller A, passes in the argument, makes the call, gets the result
  • Server B receives the argument, executes the logic, and returns the result

The RPC framework is responsible for the big blue box in the middle:

  • Client side: serialization, deserialization, connection pool management, load balancing, failover, queue management, timeout management, asynchronous management, etc

  • Server side: server side components, server side packet queue, IO thread, worker thread, serialization deserialization, etc

The server side of the technology we understand more, next focus on the client side of the technical details.

Let’s start with the “serialization and deserialization” section of the RPC-Client section.

Why serialize?

Engineers typically use “objects” to manipulate data:

class User{

​ std::String user_name;

​ uint64_t user_id;

​ uint32_t user_age;

};

User u = new User(” shenjian “);

u.setUid(123);

u.setAge(35);

However, when storing or transferring data, “object” is not so useful. It is often necessary to convert the data into a continuous space “binary byte stream”. Some typical scenarios are:

  • Disk storage of database indexes: Database indexes are stored in memory as B + trees, but this format cannot be stored directly on disk, so b+ trees need to be converted into continuous space binary byte streams before they can be stored on disk

  • Cache KV storage: Redis /memcache is a KV type cache. The value stored in the cache must be a continuous space binary stream, not a User object

  • Network transmission of data: The data sent by the socket must be a binary stream of contiguous space and cannot be an object

Serialization is the process of converting data in the form of “object” to data in the form of “continuous space binary byte stream”. The reverse of this process is called deserialization.

How do I serialize it?

This is a very detailed question. If you were to convert an object into a byte stream, what would you do? One obvious way to do this is with a self-describing markup language like XML (or JSON) :

With defined transformation rules, it is easy for a sender to serialize an object of the User class to XML, and it is easy for a server to normalize it to a User object when it receives an XML binary stream.

Voiceover: This works easily when the language supports reflection.

The second method is to implement the binary protocol for serialization, or to design a generic protocol for the User object above:

  • The first four bytes indicate the ordinal number

  • The four bytes following the sequence number represent the length m of the key

  • The next m bytes represent the value of key

  • The next four bytes represent the length n of the value

  • The next n bytes represent the value of value

  • Like XML, the recursion continues until the entire object is described

The User object above, described using this protocol, might look like this:

  • First line: The ordinal number is 4 bytes (let 0 represent the class name), the class name is 4 bytes long (length is 4), and the next 4 bytes are the class name (” User “), which is 12 bytes in total
  • Line 2: Serial number 4 bytes (1 for first attribute), attribute length 4 bytes (length 9), followed by 9 bytes for attribute name (” user_name “), attribute value 4 bytes (length 8), attribute value 8 bytes (value “shenjian”), 29 bytes in total
  • Line 3: Serial number 4 bytes (2 for the second attribute), attribute length 4 bytes (length 7), followed by 7 bytes for the attribute name (” user_id “), attribute value 4 bytes (length 8), attribute value 8 bytes (value 123), 27 bytes in total
  • Line 4: Serial number 4 bytes (3 is the third attribute), attribute length 4 bytes (length 8), next 8 bytes are attribute name (” user_name “), attribute value 4 bytes (length 4), attribute value 4 bytes (value 35), total 24 bytes

The entire binary byte stream is 12+29+27+24=92 bytes.

For example, strongly typed languages restore not only the property name and value, but also the property type; Complex objects should consider not only ordinary types, but also object nested types and so on. In any case, the serialization idea is similar.

What factors do serialization protocols take into account?

Whether you use a mature XML/JSON protocol or a custom binary protocol to serialize objects, you need to consider the following factors when designing the serialization protocol.

  • Parsing efficiency: This should be the primary consideration of serialization protocols. XML/JSON parsing is time-consuming and requires parsing of the DOOM tree. Binary custom protocol parsing is efficient

  • Compression rate, transmission efficiency: The same object, XML/JSON transmission has a large number of XML tags, information validity is low, the binary custom protocol takes up much less space

  • Expansibility and compatibility: Whether fields can be easily added, and whether the old client needs to be forcibly upgraded after fields are added, are all issues that need to be considered. XML/JSON and binary protocols above can be easily extended

  • Readability vs. debuggability: This is easy to understand, XML/JSON readability is much better than binary protocols

  • Cross-language: Both protocols are cross-language. Some serialization protocols are language specific. For example, Dubbo’s serialization protocol supports only Java RPC calls

  • Generality: XML/JSON is very common, there are good third-party parsing libraries, each language is very convenient to parse, although the above custom binary protocol can be cross-language, but each language needs to write a simple protocol client

What are the common serialization methods?

  • XML/JSON: poor parsing efficiency and compression rate, good scalability, readability and versatility
  • thrift
  • Protobuf: google-made, must be excellent, all things considered, highly recommended, binary protocol, less readable, but there is a similar to-string protocol to help debug problems
  • Avro
  • CORBA
  • Mc_pack: those who understand will understand, those who don’t understand will not understand, used in 2009, legend has surpassed protobuf in all aspects, those who know can say the status quo
  • …

RPC – client except:

  • Serialize the deserialized portion (1, 4 in figure above)

Also includes:

  • Part of sending byte stream and receiving byte stream (figure 2 and 3)

This part is divided into synchronous call and asynchronous call two ways, the following one will be introduced.

Voiceover: It is not easy to make a thorough investigation of rPC-client.

The code snippet for the synchronous call is:

Result = Add(Obj1, Obj2); // block until Result is obtained

The code snippet for the asynchronous call is:

Add(Obj1, Obj2, callback); // Returns directly after the call, with no result

The processing result is obtained through the callback:

Callback (Result){// This callback function is called when the Result of the processing is obtained

…

}

These two types of calls are implemented completely differently in rPC-client.

What is the rPC-client synchronous call architecture?

Synchronous calls, which block until the result is obtained, occupy a worker thread. The diagram above briefly illustrates the components, interactions, and process steps:

  • The large box on the left represents one of the caller’s worker threads
  • The pink box on the left represents the RPC-client component
  • The orange box on the right represents the RPC-server
  • The two blue boxes represent the two core components of synchronous RPC-client, serialization component and connection pool component
  • The small white process box, with arrow numbers 1-10, represents the serial execution steps of the entire worker thread:

1) Business code initiates RPC call:

Result=Add(Obj1,Obj2)

2) Serialization component that serializes object calls into a binary byte stream, which can be understood as a packet to be sent;

3) Get an available connection through the connection pool component;

4) Send packet packet1 to rPC-server through connection;

5) Send packets to the RPC-server for network transmission;

6) The response packet is transmitted on the network and sent back to the RPC-client.

7) Receive response packet packet2 from rPC-server through connection;

8) Put the conneciont back into the connection pool through the connection pool component;

9) Serialization component, which serializes the Packet2 norm into a Result object and returns it to the caller;

10) The business code gets the Result, and the worker thread continues down;

Voice-over: Refer to steps 1-10 in the architecture diagram.

What does the connection pool component do?

RPC frame lock supports load balancing, failover, send timeout and other features, which are realized by connection pool components.

Typical connection pool components provide external interfaces as follows:

int ConnectionPool::init(…). ;

Connection ConnectionPool::getConnection(a);

int ConnectionPool::putConnection(Connection t);

What does init do?

It establishes N TCP long connections with the downstream RPC-server (usually in a cluster), which is called connection “pool”.

What does getConnection do?

Take a connection from the connection pool, lock it (with a flag bit) and return it to the caller.

What does putConnection do?

Unlocks (also with a flag bit) an allocated connection by putting it back into the connection pool.

How to achieve load balancing?

The connection pool establishes a connection to an RPC-server cluster, and the connection pool returns the connection randomly.

How is failover implemented?

The connection pool establishes a connection to an RPC-server cluster. If the connection pool finds that a connection to a machine is abnormal, it needs to remove the connection from the machine and return to the normal connection. After the machine recovers, the connection is added back.

How to implement send timeout?

When a connection is received, send/ RECV with timeout can be used to send and receive the connection with timeout.

In general, the implementation of synchronous RPC-client is relatively easy. Serialization components, connection pool components, and multiple worker threads can be implemented.

Remaining questions, what is the best number of worker threads to set?

This question is addressed in “What is the optimal number of worker threads?” I will not go into the details here.

What is the rPC-client asynchronous callback architecture?

Asynchronous callbacks do not block until the result is obtained. Theoretically, no thread is blocked at any time. Therefore, the model of asynchronous callbacks theoretically requires very few worker threads to connect to the service to achieve high throughput, as shown in the figure above:

  • The box on the left is a small number of worker threads (a few will do) making calls and callbacks
  • The pink box in the middle represents the RPC-client component
  • The orange box on the right represents the RPC-server
  • The six blue boxes represent the six core components of asynchronous RPC-Client: context manager, timeout manager, serialization component, downstream send/receive queue, downstream send/receive thread, connection pool component
  • The small white process box, with arrow numbers 1-17, represents the serial execution steps of the entire worker thread:

1) Business code initiates asynchronous RPC call;

Add(Obj1,Obj2, callback)

2) Context manager, which stores requests, callbacks, and contexts;

3) Serialization component, which serializes the object call into a binary byte stream, which can be understood as a packet to be sent;

4) The downstream sending and receiving queue puts the packet into the “waiting queue”. At this time, the call returns without blocking the working thread;

5) The downstream sending and receiving thread takes out the message from the “waiting queue” and gets an available connection through the connection pool component;

6) Send packet packet1 to rPC-server through connection;

7) Send packets to the RPC-server for network transmission;

8) The response packet is transmitted on the network and sent back to the RPC-client;

9) Receive response packet packet2 from rPC-server through connection;

10) The downstream sending and receiving thread puts the packet into the “Accepted queue” and puts the conneciont back into the connection pool through the connection pool component;

11) When the packet is taken out in the downstream sending and receiving queue, the callback will start without blocking the working thread;

12) Serialization component, serialize packet2 norm to Result object;

13) Context manager, fetching results, callbacks and contexts;

14) Callback business code through callback, return Result, the worker thread continues down;

If the request is not returned for a long time, the process is as follows:

15) Context manager, the request does not return for a long time;

16) The timeout manager gets the timeout context;

17) Timeout_cb calls back the business code and the worker thread continues down;

Voiceover: Please look at this process several times with the architecture diagram.

Serialization components and connection pool components, as described above, are easier to understand than sending and receiving queues and threads. The two overall components of the context manager and timeout manager are highlighted below.

Why do YOU need a context manager?

Since the request packet is sent, the callback of the response packet is asynchronous and not even done in the same worker thread. A component is required to record the context of a request and match the request-response-callback information.

How do you match request-response-callback information?

A request packet a, b, and C was sent to the downstream service through a connection, and three response packets X, Y, and Z were received asynchronously:

How do I know which request package corresponds to which response package?

How do I know which response package corresponds to which callback function?

The request-response-callback concatenation can be achieved through the request ID.

The request ID and context manager correspond to the request-response-callback mapping:

1) Generate request ID;

2) Generate the request context, including sending time, callback and other information;

3) The context manager records the mapping relationship between req-ID and context;

4) Type the REQ-ID in the request packet and send it to the RPC-server;

5) The RPC-server typed the REq-ID into the response packet and returned it;

6) Find the original context context by the req-ID in the response package through the context manager;

7) Retrieve the callback function from the context;

8) Callback brings back Result to promote further execution of business;

How to achieve load balancing and failover?

The idea is similar to synchronous connection pooling, except that:

  • Synchronous connection pool Sends and receives packets in blocking mode. Multiple connections must be established with one IP address of one service

  • Asynchronous sending and receiving, where only a few connections (for example, a TCP connection) need to be established for each IP address of a service

How to implement timeout send and receive?

Timeout sending and receiving is implemented differently from synchronous blocking sending and receiving:

  • Synchronous block timeout can be implemented by sending/RECV with timeout

  • Asynchronous non-blocking niO network packet sending and receiving, because the connection does not wait forever for a packet back, the timeout is implemented by the timeout manager

How does the timeout manager implement timeout management?

Timeout manager, which is used to implement timeout callback processing for packet callback requests.

For each request sent to the downstream RPC-server, req-ID and context information will be saved in the context manager. The context stores a lot of information about the request, such as REQ-ID, callback, timeout callback, send time, etc.

The timeout manager starts the timer to scan the context in the context manager to see if it takes too long for the request to be sent in the context. If it takes too long, it does not wait for the packet to be sent back, and simply timeout the callback to push the business process forward and remove the context.

If a normal callback arrives after the timeout callback is executed, the request is discarded if the context is not found in the context manager through req-ID.

Voice-over: Unable to recover context because it has timed out.

However, asynchronous callbacks and synchronous callbacks have more context managers, timeout managers, downstream send/receive queues, downstream send/receive threads, and other components than serialization components and connection pool components, and have an impact on the calling habits of callers.

Voiceover: Programming habits change from synchronization to callback.

Asynchronous callback improves the overall throughput of the system. The specific rPC-client implementation mode can be selected based on service scenarios.

conclusion

What is an RPC call?

Call a remote service as if it were a local function.

Why do YOU need an RPC framework?

The RPC framework is used to mask the serialization, network transmission and other technical details during RPC calls. Let the caller only focus on the invocation and the service only focus on implementing the invocation.

What is serialization? Why serialization?

The process of converting an object into a continuous binary stream is called serialization. Disk storage, cache storage, and network transport can only operate on binary streams and must be serialized.

What are the core components of synchronous RPC-client?

The core components of synchronous RPC-client are serialization component and connection pool component. It realizes load balancing and failover through connection pooling and timeout processing through blocked sending and receiving.

What are the core components of asynchronous RPC-client?

The core components of asynchronous RPC-client are serialization component, connection pool component, send/receive queue, send/receive thread, context manager and timeout manager. It associates request-response-package-callback functions with “request ID”, manages context with context manager, triggers timeout callback with timer in timeout manager, and advances timeout processing of business process.

Thinking is more important than conclusion.