System design, protocol first.
Most people do not know the details of protocol design and use existing protocols for application layer design. For example:
(1) Use HTTP, design get/ POST /cookie parameters, and JSON package format;
(2) Use dubbo instead of delving into the details of the internal binary header body;
However, understanding the principles of protocol design is very helpful in understanding system communication.
The so-called “agreement” is the rules that both parties abide by, such as: divorce agreement, armistice agreement. The protocol has three elements: syntax, semantics and timing: (1) syntax, namely the structure or format of data and control information; (2) semantics, that is, what kind of control information should be sent, what kind of action should be completed and what kind of response should be made; (3) timing sequence, that is, the detailed description of the event realization sequence;
Voice-over: The rest of the article focuses on grammar design.
Protocol design is usually divided into three layers:The application layerThe agreement,Security layerThe agreement,The transport layerThe agreement.
The following is how to select the protocol of the three layers.
2. Application layer protocol design
There are three common protocols for application layer selection: text protocol, binary protocol and streaming XML protocol.
Text protocol
A text protocol is a communication transmission protocol close to human written language. A typical protocol is HTTP. The following is an example of an HTTP request packet:
GET / HTTP/1.1
User-Agent: curl
Host: musicml.net
Accept: /
The text protocol has the following characteristics: (1) good readability and easy debugging; (2) Good scalability and can be extended through key:value; (3) The parsing efficiency is not high. The key and value are read line by line and separated by colons. (4) Not friendly to binary, such as voice/video;
Binary protocol
Binary protocols are typically IP protocols. The following is an illustration of IP protocols:
Binary protocols generally include:
(1) Fixed-length Baotou;
(2) expandable variable length inclusion;
(3) Generally, each field has a fixed meaning. Take IP protocol as an example, the first four bits indicate the protocol Version number.
The binary protocol features are:
(1) Poor readability and difficult to debug;
Voice-over: Logging generally requires a toString() function to improve readability.
(2) Poor scalability, if you want to extend the field, the old Version protocol is not compatible, so there is usually a Version field in the design; (3) High parsing efficiency, almost no parsing cost, binary stream each field represents a fixed meaning; (4) Natural support for binary streams, such as voice/video;
Here is a typical example of a 16-byte binary fixed-length header:
//sizeof(cs_header)=16
struct cs_header {
uint32_t version;
uint32_t magic_num;
uint32_t cmd;
uint32_t len;
uint8_t data[];
}attribute((packed));
Among them:
(1) The first four bytes represent the version number.
(2) The next four bytes represent the magic number magic_num, which is used to solve data misplacement or packet loss problems;
Voice-over: For example, if the magic number is 0x01020304, the received packet is regarded as normal if the magic number matches. Otherwise, the packet is regarded as abnormal and the connection is disconnected.
(3) The next four bytes represent the command number. Different command numbers correspond to different variable length packages. (4) The last four bytes represent the packet length to determine how many bytes there are in the variable length packet;
Here is an actual binary variable length package:
message CUserLoginReq {
optional string username = 1;
optional string passwd = 2;
}message CUserLoginResp {
optional uint64 uid =1;
}
It uses Google’s Protobuf protocol and is easy to see:
(1) The user name and password are transmitted in the request packet.
(2) The response package returns the user’s UID;
PB is a very popular binary variable length package protocol, its advantages are: (1) universal, can generate C++, Java, PHP and other languages code;
(2) Built-in compression function;
(3) Binary friendly;
(4) it has been widely used in industry; Voice-over: Google produced, must be fine.
Streaming XML protocol Streaming XML seems to be a special case of text protocols and can be a separate class. For example, XMPP is a typical streaming XML protocol. Here is a typical MESSAGE for XMPP:
<message
To = ‘[email protected]’
The from = ‘[email protected]’
Type = “chat”
XML: lang = “en” >
Wherefore art thou, Romeo?
From the XML tag, you can roughly determine that this is a chat message sent by Romeo to Juliet.
XML protocol has several characteristics: (1) good readability and scalability, which are the characteristics of XML; (2) The parsing cost is extremely high and DOM tree analysis is required;
(3) The effective data transmission rate is extremely low and there are a large number of tags;
(4) Not friendly to binary, such as voice/video;
Iii. Security layer protocol design
Security layer protocol design, in addition to the use of SSL, self-implementation, and common there are the following three schemes.
Voice-over: SSL key management is a problem.
Fixed key The server and the client agree on a key and an encryption algorithm (for example, AES). Each time the client sends a packet, the server encrypts the packet using the algorithm and the key. After receiving the packet, the server decrypts the packet using the algorithm and the key.
Voiceover: Low security, security based on programmer’s professional ethics.
One person one secret simply means that one person’s key is fixed, but different from one person to another. Common implementations are:
(1) Fixed encryption algorithm;
(2) Add secret key use “a special attribute of the user”, such as user UID, mobile phone number, QQ number, user password, etc.;
One key at a time is a dynamic key. One key per Session is more secure. The key is negotiated before each Session. The key negotiation process requires two random generation of asymmetric keys and one random generation of symmetric encryption keys. The details are not expanded here.
Four, transport layer protocol design optional protocols are TCP and UDP, now basically use TCP, with epoll and other technologies, multi-connection is not the bottleneck, there is no problem with hundreds of thousands of links in a single machine.
The Architect’s Path – Share practical technical articles
Related recommendations:
The Architect’s Journey of 18 Years
Research:
Your company’s encryption, do you write dead keys in code?