“This is the second day of my participation in the August More Text Challenge.

Remember before

In the first half of 2019, the project needed RPC to establish connections between different services, so it began to investigate RPC-related content. Since the project itself was based on Asyncio ecology, it began to look for RPC framework based on Asyncio ecology. At that time, there were few RPC frameworks for Asyncio in Python. There was only one AIORPC, which fulfilled the basic functions of RPC but lacked the service governance functionality that other RPC frameworks had (would it be better called service Management??). , so at that time, I wanted to start building a Python Asyncio ecological RPC framework — RAP, this project in addition to meeting the functions of RPC in any language (although other languages are not written…). , but also meet fast; Simple core functions; Can provide high-level functionality for Python alone; It can be used independently without other services, and it can be extended in the form of plug-ins.

Note 1: Asyncio has some advantages and disadvantages

Note 2: original address

1. What does the RPC framework require

There is no doubt that the most important part of RPC framework is RPC. The full name of RPC is remote procedure call. Its essence is that the client converts the local call of the user into a message and sends it to the server through a connection. The process of finally returning the results to the client and back to the user by the client.

The RPC framework encapsulates the above call process, allowing callers to call remote functions as local functions through the RPC framework without paying attention to the low-level details. So what does this call require? I start by imagining the simplest business — the user calls a function foo:

def foo(a: int, b: int) - >int:
	pass
Copy the code

Then the invocation of this business is transformed into an imagined RPC request flow, as shown below:It can be seen that THE RPC framework needs to convert the function signature called according to the application layer protocol and transfer the data to the server through connection. After receiving the request, the server extracts the information according to the application layer protocol, searches for the corresponding function according to the information, and then invokes it. Finally, the generated result is returned to the client, which processes it and then returns it to the user. In addition, serialization and deserialization are essential because information is transmitted. From the whole process, it can be found that an RPC framework should at least have application layer protocol, serialization and deserialization, and network transmission. After completing these points, a simplest RPC framework can be written.

2. Transport layer protocol

At present, the protocols of RPC framework are generally a combination of multiple protocols, including transport layer protocols, application layer protocols, etc. (OSI model is adopted for layering, the same below). At present, there are two kinds of transport layer protocols, one is TCP, it transmits stable and reliable, can try to ensure that information does not lose packets. The other is UDP, which is highly customizable and has low performance overhead, but may lose packets across networks. However, THE APPLICABILITY and availability of RPC framework are both strong, so the current TRANSPORT layer protocol of RPC framework is TCP (UDP is only considered if it is used in Intranet network), and I also choose TCP as my transport protocol.

3. Application layer protocol

Through investigation, it is found that currently open source RPC framework either develops its own application layer protocol based on TCP protocol, or directly uses HTTP as its application layer protocol. These two methods have their own advantages, choose HTTP as the application layer protocol, can have a mature ecology, can use Nginx and other middleware, but HTTP request will transmit a lot of fields, some fields may never be used, will waste the transmission performance, and choose their own customized application layer protocol. It can not use some mature middleware, but it can keep the request body simple as far as possible. After comprehensive consideration, I choose to customize the application layer protocol.

Before customizing application layer protocols, we need to figure out which functions need to be implemented based on application layer protocols. After reviewing some RPC frameworks, I determined several basic functions:

  • one-by-oneFormal sending and receiving messages: This function is used for the most basic sending and receiving messages.
  • Duplex message sending and receiving: Currently all RPC frameworks in the market support this type of message, HTTP also supports duplex message sending and receiving through WebSocket, I also want my framework to support this type of message, and named asChannel
  • Control connection status messages: In order to keep the connection stable and detect errors in a timely manner, you need to passKeep-AliveThis type of message is independent of the message invoked by the user.
  • Speed up transmission: Since RPC framework will bear a large amount of traffic, while ensuring user performance and low enough response time, it is necessary to speed up transmission as much as possible without increasing too much system occupation.

After confirming the basic functions, you can start to develop the protocol. A good protocol must have good expansion and simplicity. Because I am also the first time to customize private protocol, I am still exploring, so I draw some inspiration from HTTP protocol (a small part from AMQP protocol).

The first is HTTP/1, the most commonly used FORM of HTTP request, one-by-one, which only requests and responds once. If not optimized, each request goes through four stages: creating TCP, making the request, reading the response, and destroying TCP. TCP creation and destruction is a huge waste of performance and adds a lot of time. Therefore, HTTP began to support connection multiplexing, also known as persistent connections, which allows clients to hold one or more TCP connections to the same domain name for a long time. Typically, clients maintain a FIFO queue. The system does not automatically disconnect data for a period of time after the data is fetched. Instead, the system puts the data back to the queue for direct reuse when the next resource is fetched, eliminating the cost of TCP creation and destruction. Connection reuse, however, this technique is not perfect, it is one of the most obvious side effects team first blocking problems, such as A and B have two requests, they use the same TCP connection, when A request is sent and the corresponding service side has been no response, B requests will have to wait, until I received A request of the TCP connection. In fact, A TCP connection can send A request and B request almost at the same time, and can receive A response and B response almost at the same time, but HTTP/1.0 does not recognize which response is generated by that request. In HTTP/2, a frame is the smallest unit of information. It can be used to describe various data, such as Headers and Body of a request, or to indicate control, such as opening or closing a connection. At the same time, each frame is attached with a stream ID to identify which stream the frame belongs to, so that the client can match different responses and requests based on the ID, eliminating the problem of recognizing that the same connection receives two requests at the same time. One of the most important technical features of this design of HTTP/2 is called HTTP/2 multiplexing.

Getting used to the one-by-one approach to HTTP/2 multiplexing can be hard to shift from. When I understand it, I think of an HTTP connection as a river, where each request is equivalent to a shipment, and the sender sends the shipment with his own label and sends it to a ship. Some ships will carry goods from point A to point B, and some ships will carry goods from point B to point A. They don’t care what the goods are, but the goods have their own mark, according to which the port can know the goods are issued by the request, which request to recover.

From the iterative history of HTTP protocol, it can be found that HTTP is gradually improving the reuse of TCP connections, so as to reduce the waste of resources. However, for RPC, it naturally requires excellent transmission performance and less machine cost. At the same time, the connection created when the client establishes the connection will always exist. The connection is not disconnected until one of the parties fails for some reason, and all requests are transferred over the connection. The scenario is very simple, so it requires more multiplexing.

So how do you implement multiplexing? The steps are as follows:

  • The requesting function allocates a unique label before making the request, then creates an empty space in a shared container to wait for the data to arrive, and then puts the label in the request body according to the protocol and sends it to the server.
  • When the server receives the request and generates a response, it puts the label in the response body according to the protocol and returns the data to the client.
  • Client connection will start when creating a background task, the task will read data from the connection, if you have the data, will parse the data, according to the agreement to extract the labels and put the data in the location on the container, if not found the corresponding position you will need to discard data (prove the connection did not send the label data).
  • The requesting function waits from the empty location until the data arrives, returns it to the user, and deletes the location to prevent overflow.

The following is a simple pseudo-code description (specific source can be seen: github.com/so1n/rap/bl… :

# pseudocode
Select id, header, and body
# The response body transmits the id, header, status_code, and body
The id of each request is used to identify which request belongs to
Make sure each request has a different ID to reuse the connection

A dict holds future(asyncio.future ()) that sent a request but received no response.
future_dict = dict(a)async def response() :
    # Unified processing response, there will be a program to run in the background
	while True:
    	try:
    		Fetch response data from the connection according to the protocol
    	    msg_id, header, status_code, body = await conn.read() 
    	except ValueError:
    	    return
    	Store the result into the future so that the request can receive the response
		if msg_id in future_dict:
    		future_dict[msg_id].set_result(Response(msg_id, header, status_code, body))


async def request() :
	# request body
    request = (msg_id, header, body)
    try:
        # set the future waiting for the response
        future_dict[msg_id] = asyncio.Future()
		try:
	        # indicates that the request is sent over the conn connection
	        await conn.write(request)
	    except Exception as e:
	        raise e
        try:
            If there is no response after 9 seconds, the request will timeout
            return await asyncio.wait_for(future_dict[msg_id], 9)
        except asyncio.TimeoutError:
            raise asyncio.TimeoutError(f"msg_id:{msg_id} request timeout")
    finally:
        # delete future, recycle resource
		future_dict.pop(msg_id, None)
Copy the code

After understanding multiplexing, we can find that there is an essential field in the protocol, which is the message label mentioned above, which I named as the correlation ID in the protocol. In addition, when understanding connection multiplexing, I also incidentally understand the design of HTTP protocol. Through the design of HTTP protocol, I extracted a few fields from it and finally consolidated them into the following fields:

  • Message ID: This message ID is monotonically increasing and allows both parties to know if a message has been retransmitted and to expand for future functionality.
  • Protocol version: Generally, protocols change with iteration. Therefore, a field is required to identify the current protocol version to facilitate client and server identification. Also, this field should be at the top of the protocol as much as possible, because it is the least likely to change.
  • Message type: As you can see from the design of HTTP/2, HTTP/2 sends a separate request to control opening and closing a connection, in addition to sendingkeep aliveRequest to maintain communication with the server. This type of request is different from normalone-by-oneRequests are different. And I also want to have typewebsocketIt is also different from the above two request types, so a field is needed to distinguish their types, the specific types are as follows:
    • event: This type of message is used to control the connection status and behavior of the transmission. This type of request allows both the client and the server to send and receive the request, for exampleping.pongAnd determine the connection lifecycledeclare.msg.dropAnd so on.
    • msg: This type of message is very similar to the traditional HTTP/1, and is the most classic and common message model. It only accepts a send and receive, and the correlation id of the send and receive must be the same
    • channel: This type of message can be similar toWebsocket, is a message model that can be sent and received both ways at the same time. It can also be changed to one-shot multi-collect or multi-shot single-collect, as defined by the implementer himself.
  • Correlation ID: This value exists to enable multiplexing as described above, and the client can use the correlation ID to find out which request the response is a response to.
  • Destination URL: This field is like an HTTP URL that the client can use to find the routing function corresponding to the server. However, for RPC, URL does not need to be that long. Generally, the target function of RPC is searched by which group and which function in the group, so THE URL will be/{group}/{func name}To represent.
  • Header: The protocol is designed to be both compact and easy to extend. Headers in HTTP make it easy for users to customize extensions, so I used the header field
  • Body: This field stores the body of the response and request, also similar to http.body.
  • Status code: Similar to HTTP, this field is available only for response. The value less than 300 is a normal request, the value greater than 300 and less than 400 is an extension of function, the value greater than 400 and less than 500 is an exception request, and the value greater than 500 is a destructive exception.

After the required fields are determined, they can be arranged into a protocol. According to the principle of simplicity and less change, the final request message protocol is defined as:

# message ID, protocol version, message type, correlation ID, target URL, header, body MSg_id, version, msg_type, correlation_id, target, header, bodyCopy the code

The response message protocol is defined as:

# message ID, protocol version, message type, Correlation ID, target URL, status_code, header, body MSg_id, version, MSg_type, correlation_id, target, status_code, header, bodyCopy the code

Now, the application layer protocol is finally customized, but it is not enough, because the program in the read and write protocol is actually still exist in memory data structures or objects, while network transmission only transmits binary code, and the expression of object types in different languages is not the same. Therefore, there is a need for a general method of converting data structures or objects to binary encoding — serialization, and the reverse — deserialization.

4. Serialization and deserialization

Serialization and deserialization are the most important aspects of network transport, and are essentially a protocol, presumably at the presentation layer of the OSI model. Json is the serialization protocol that most people come into contact with. It is universal, mature and convenient to debug, but it takes up a lot of space and takes a long time to serialize, thus affecting the performance of transmission. So I didn’t pick Json as my serialization protocol (I knew from working with the Starlette framework that there were two libraries in Python, uJSON and OJSON, which were also Json in nature, but had a way of reducing the CPU performance impact of serialization). However, I was not able to develop a mature and stable serialization protocol, so I had to choose from the open source protocol. Each of the open source serialization protocols has advantages and disadvantages, and they have their own unique application scenarios at the beginning of design. While choosing the serialization protocol, I made analysis based on the following points:

  • Versatility: this is whether the protocol supports cross-platform, cross-language support, but not support, then the versatility is greatly reduced.
  • Epidemic: epidemic mainly is to use the person is not much, if the person is not much, the data will be less, the speed of the solution pit will become slow.
  • Maturity: Whether the protocol is mature. Some serialization protocols have strong performance but few functions when they were born, and are unstable and insecure. If you want to make a stable RPC framework, the serialization protocol used must be mature and stable.
  • Space occupation: If the space occupied after serialization is still too much, it will increase the transmission pressure of the network and reduce the response time.
  • Time footprint: If the serialization protocol is more complex, it takes longer to parse and consumes more CPU time as I usePython asyncioFor ecological development, complex agreements will occupy more CPU time and reduce QPS, so such agreements cannot be selected.

In addition to readability, people can easily read the serialization protocol with high readability, which will be very convenient for debugging, but increasing readability will inevitably lead to large space consumption and CPU parsing time, so this feature is not included in my consideration.

After a search, I foundmsgpackThe serialization protocol, which is a binary Json serialization protocol, is similar to Json, but with some differences, as shown in the following figure:This is a screenshot from the official website, which introduces a data structure after serialization of Json andmsgpackIt can be seen that it is designed for network transmission. The serialization result itself is very compact and its volume is greatly reduced, which can reduce the traffic of network transmission and improve the performance of transmission. At the same time, according to its documents, it can be used in the same way as Json. It works pretty much like Json to the user, doesn’t cost much to learn, and is a perfect match for my needs.

Some insights into serialization, and some serialization comparisons see serialization and deserialization

The MaspCAk benchmark can be found at: github.com/alecthomas/… Or: github.com/endel/msgpa…

5. To summarize

After investigating transport protocols, serialization protocols, and customizing your own transport protocols, you can start writing a simple RPC framework, but showing all the code would take a lot of space, so here’s a simple request flow, starting with the connection lifecycle flow:Then there is the flow for each request (much like the flow envisioned in the beginning) :

As you can see, there are some hooks in the diagram in addition to what is described above, this is because the diagram only completes a basic RPC framework, which only meets the simplest remote procedure call function. However, RPC involves network traffic, and there is a lot of uncertainty, so the framework also needs some service governance-related functions, and most of these functions will be extended through hooks, which will be introduced later, some in isolation, some in combination.