Welcome to reprint, reprint please indicate the source: juejin.cn/post/685003…
Writing in the front
Finally, I can start to write this series of articles. This series of articles is expected to be divided into 13 parts. Since the knowledge points involved in IM are a little complicated, each knowledge point will be explained in a separate article to make it as thorough as possible for everyone to understand.
The soul of torture
- Why write this series of articles?
You may ask, with the previous NettyChat and open source a self-use Android IM library, based on Netty+TCP+Protobuf implementation, why still need to write this series of articles? The main reason is that when NettyChat was first opened and published, it was intended to be a good introduction to IM. And one article is difficult to explain all the knowledge points, plus NettyChat is also a Demo, some of the code is not rigorous. For more than a year, from the feedback of the group, the comments of the article can see that people have a lot of demand for this piece of knowledge, people to integrate NettyChat into their own projects is also more trouble, there is also a lack of complete IM implementation online, most of the knowledge is piecemeal. So I’m going to start from scratch and teach you how to implement your own IM system.
- What new features and optimizations have been added to NettyChat?
Support TCP/WebSocket, Protobuf/Json, etc. Optimize the message resend manager instead of a message per Timer implementation (which is a huge waste of resources and affects program performance). In addition, optimize the code structure, improve scalability, maintainability and so on.
- Does the project contain server-side code?
This is an important reason to rewrite this series of articles, which will include Java server code and Android code, with more on IOS at a later time. After this article is completed, we will open source three projects on Github, which are:
- ims_kula[Android IM Service SDK]
- Kulachat_server [Java server]
- KulaChat[Android client]
- Why is the project called “Kula”?
On a whim, Kula is one of the author’s favorite characters in the SNK Series. Attached is Kula hd:
Outline as shown below:
This series of articles will include:
- Technical selection and protocol definition
- Interface definition and encapsulation
- Custom message codec for sticky packet/sticky packet processing
- Connection and reconnection of long connection stability
- heartbeat
- Message retransmission mechanism
- Offline messaging mechanism
- Message order control
- Database table design
- Single chat implementation
- Group chat implementation
- System message implementation
- The UI encapsulation
This paper is the first part: technology selection and protocol definition.
The project architecture
First, talk about the overall architecture of the project and the open source framework used. According to the suggestions of the friends in the group and their familiarity with Java and Kotlin, Java is still the development language for Android. Later, we will consider developing a version with Kotlin. Since the project is not completed, we are currently writing code while writing articles. Therefore, after the completion of subsequent projects, we will write a special article to introduce the project architecture, including Android client and Java server.
The Overall Android client project adopts MVP architecture and uses the following open source libraries:
- RxJava 2.2.8
- RxAndroid 2.1.1
- Retrofit 2.8.1
- GreenDao 3.3.0
- Dagger 2.27
- Butterknife 10.2.1
- Glide 4.11.0
- StatusBarUtil 1.5.1
- BaseRecyclerViewAdapterHelper 3.0.4
- AutoSize 1.2.1
- Fastjson 1.1.71. Android
- LogginInterceptor 3.1.0 – rc5
- RxPermissions 0.10.2
- Toasty 1.4.2
- Background 1.6.5
- EventCenter 1.0.1
- NiceImageView 1.0.5
- More…
The Java server is still under development, so I won’t list them all. Thanks to the authors of the open source library above.
The following communication protocol selection, transmission protocol selection, communication framework selection inOpen source a self-used Android IM library, based on Netty+TCP+Protobuf implementationThis has been explained in the article, but since I plan to write a new article in this series, I will explain it again here, first to add my new understanding, and second to integrate into this series of articles.
The following principles and theoretical knowledge, due to space reasons and the author’s level is limited, can not elaborate. The focus of this series of articles is to teach you how to implement your own IM system. As for the theoretical knowledge, the author helps you find a link to the original text. If you are interested, you can jump to the original text to read. Thanks also to the authors of the following articles.
Selection of communication protocol
Common communication protocols are as follows. This section describes the advantages and disadvantages of each communication protocol and its application scenarios.
-
UDP
- A simple overview
UDP is a packet-oriented, connectionless protocol. That is, the two parties do not need to establish a connection before sending data, and do not need to disconnect the connection after sending data (there is no connection to disconnect). This reduces the overhead of connecting and disconnecting (instead of the three handshakes required for connecting and the four waves required for disconnecting like TCP). In addition, UDP does not have a congestion mechanism, that is, congestion does not reduce the sending rate of the source host. In addition, UDP is a big feature of maximum delivery (that is, not guaranteed delivery), that is, there may be packet loss, out of order and so on. In general, UDP is an unreliable protocol.
- advantages
- High efficiency
UDP does not need to establish a connection before sending data, and does not have the confirmation mechanism, retransmission mechanism, and congestion mechanism of TCP. Therefore, the data transmission efficiency is high.
- Low overhead
The UDP header cost is only 8 bytes (source port [16bit], destination port [16bit], length [16bit], checksum [16bit]).
- A security
UDP does not have the various mechanisms that TCP does, so it is less vulnerable to attacks, but it is not immune to attacks.
-
Supports broadcast, unicast, and multicast
-
disadvantages
-
unreliable
Because UDP does not have the reliability mechanism of TCP to ensure transmission, it is easy to lose packets when the network quality is poor. Therefore, when using UDP as the communication protocol, you need to ensure reliability, such as confirming retransmission. QQ is said to use UDP as the communication protocol in the early stage, on the basis of UDP, to achieve similar TCP reliability assurance, so that it can achieve high speed transmission, but also take into account the reliability. Why does QQ use UDP instead of TCP? If you are interested, take a look.
-
out-of-order
-
Applicable scenario
UDP can be used when the network communication quality is not high and the real-time performance is high. Such as:
- Real-time audio and video chat, this situation loses some frames has little impact, does not need to retransmit, high transmission speed requirements.
- Remote control, lost some instructions do not affect.
-
TCP
- A simple overview
TCP is a byte stream – oriented, connection-oriented full-duplex protocol. Before communicating with each other, you need to establish a connection through a three-way handshake to ensure that both parties are capable of sending and receiving data. TCP is a full-duplex protocol. After data is sent, each direction needs to be closed separately. Therefore, you need to wave four times to disconnect the TCP connection. In addition, TCP provides mechanisms such as acknowledgement, timeout retransmission, sliding window, flow control, slow start, and congestion mechanism to ensure data order and reliability. In general, TCP is a reliable protocol.
- advantages
- reliable
TCP uses a series of mechanisms to ensure the reliability of data transmission, such as acknowledgement, timeout retransmission, checksum, maximum message length, sliding window mechanism, etc.
- The orderly
When TCP protocol to send data, for each shard will do order, also is a sequence number to each packet distribution, and in a specific period of time waiting for the receiving host to confirm distribution of the serial number, if the sending host within a specific time has not yet received the receiving host to confirm, then send the main opportunity retransmission packets. The receiving host uses serial number to confirm the received data, so as to detect whether the data sent by the other party is lost or out of order. Once receiving the sequential data, the receiving host reorganizes the data into data flow in the correct order and transmits it to the higher level for processing.
- disadvantages
- Low efficiency
TCP establishes a connection through three-way handshake before sending data, and disconnects the connection through four-way handshake after sending data. In addition, a series of mechanisms are required to ensure data reliability and sequence during data transmission. Therefore, the transmission efficiency of TCP is lower than that of UDP.
- Spending big
The TCP header cost is a minimum of 20 bytes (source port [16bit], destination port [16bit], sequence number [32bit], response sequence number [32bit], TCP header length [4bit], Reserved [3bit], control code [6bit], window size [16bit], offset [16B] It], checksum [16bit], options [variable length, up to 40 bytes] (optional))
- Applicable scenario
Scenarios that require high network communication quality, low transmission efficiency, and accurate data transmission. For example: financial, social apps and so on.
Note: The author has found some very good articles for you to read if you are interested:
- Network Foundation: TCP protocol – How to ensure transmission reliability
- How does TCP ensure that messages arrive in order and reliably
- In-depth understanding of TCP: from principle to Practice
-
WebSocket
- WebSocket is a protocol implemented based on TCP, which is a TCP long connection application. The difference is that WebSocket handshake uses Http request. After connecting to the server successfully, the client initiates an Http handshake request to the server. After receiving the handshake request and passing the verification, the server returns a Response to the client. After the handshake is complete, the data is sent over TCP, which means the WebSocket only uses Http to complete the handshake. In general, WebSocket is a protocol implemented and encapsulated based on TCP.
- advantages
Because WebSocket is a protocol implemented based on TCP, it has the same advantages as TCP. In addition, the standard implementation of WebSocket that follows THE RFC specification and has its own packet length does not produce sticky packets/unpackets. Therefore, it can even be considered that one of the functions of WebSocket is to deal with TCP sticky packets. Of course, not all WebSocket implementations follow the RFC specification, so in extreme cases, you have to deal with it yourself. Websockets need to subcontract logical packets like TCP sockets. In addition, WebSocket on the basis of TCP, encapsulated some frame implementation, convenient for us to use.
- disadvantages
With TCP.
- Applicable scenario
With TCP. In addition, from the WebSocket name can be seen, more suitable for Browser’s instant messaging implementation.
Note: WebSocket, like HTTP, is an application layer protocol based on TCP. The handshake section is designed to be compatible with existing HTTP-based server components (Web server software) or middleware (proxy server software). This allows a port to accept both regular HTTP and WebSocket requests. For this purpose, the WebSocket client handshake is an HTTP Upgrade Request. To be interested, read the following articles:
- WebSocket(1) a brief introduction to handshake connections
-
MQTT
- A simple overview
MQTT is a “lightweight” communication protocol based on a publish/subscribe model, which is also an application of TCP long connections (with a layer of Websockets in between, of course). Why do WE need MQTT when we have TCP and WebSocket? This is because MQTT has the great advantage of providing real-time and reliable messaging services for connecting remote devices with minimal code and limited bandwidth, as well as the choice of three quality of message publishing (QoS Level) depending on the network environment:
- QoS0, At most once
- QoS1, At least once
- QoS2, Exactly once, make sure it’s only once
As an instant messaging protocol with low overhead and bandwidth consumption, it is widely used in Internet of Things, small devices, mobile applications and so on. In MQTT protocol, an MQTT packet consists of Fixed header, Variable header and payload.
- Fixed headers. It exists in all MQTT packets and represents the packet type and packet grouping class identification.
- Variable headers. Exists in partial MQTT packets, and the packet type determines the presence and content of the variable header.
- Payload is the message body. Exists in partial MQTT packets and represents specific content received by the client.
In general, MQTT is an application layer protocol on top of TCP, which makes a lot of optimization for the Internet of Things application environment. TCP transport layer protocol is a more general layer protocol.
- advantages
Since MQTT is a protocol implemented based on TCP, it has basically the same advantages as TCP. At the same time, MQTT is also a standard RFC protocol, which is more standard than the private protocol, and has the following advantages:
- The protocol is perfect and ready for immediate production. After each end implements the same protocol, it can communicate with each other. Proprietary protocols also require extensive validation to see if they are flawed or ill-considered.
- Standardization of protocols has led to a large number of open source components, making development easier. As the ecosystem of iot +5G gets better and better, there are more and more open source components, which can reduce the amount of repeated coding.
- Standard protocols facilitate third-party access. When a third-party device or platform wants to connect, throw out a standard MQTT protocol and slap them in the face, and no one has a reason to change the interface anymore.
- There are many open source brokers, which are easy to access, etc.
Of course, this could be done with TCP’s own protocol, so why MQTT? In fact, MQTT also realizes many functions, reducing the development complexity, such as: heartbeat mechanism, asynchronous mechanism, will message, subscription publishing mechanism, QoS message quality, etc., and MQTT has made some optimization, such as the minimum message header is only two bytes, etc. Therefore, it can be simply understood that MQTT is actually a encapsulation implementation of TCP protocol, which has made a series of optimizations on the basis of TCP and encapsulated many practical mechanisms. In a word, MQTT is the network amplification version of observer mode.
- disadvantages
With TCP. In addition, although MQTT encapsulates many mechanisms, it is still immature and complex to implement.
- Applicable scenario
- IoT IoT
- Instant Messaging IM
- Embedded development equipment (not often connected or with poor network environment)
- push
- Internet of Vehicles platform
- Other scenarios with low protocol overhead
Note: Please read the following articles for interest:
- Read MQTT protocol
- Introduction to MQTT
- Advantages of MQTT over TCP long connection transparent transmission
Selection of transport protocol
Common transport protocols are as follows. This section describes the advantages and disadvantages of each transport protocol and its application scenarios.
- JSON
- A simple overview
JSON is a data format designed for data exchange. It is stored in key/value pairs. I believe that we have had many contacts in the development of peacetime, I will not introduce.
- advantages
- Readability
- More compact, good transmission efficiency
- Easy to parse
- disadvantages
- Transmission efficiency is not high (there are redundant symbols, such as “:”, space, etc.)
- Applicable scenario
Most scenarios that require data interaction can be used.
- XML
- A simple overview
XML stands for Extensible Markup Language, which I won’t cover here.
- advantages
- The format is unified and complies with standards
- High readability and self-description
- Simple, high scalability
- disadvantages
- There is a lot of redundant information that is bulky and useless
- Difficult to parse
- Applicable scenario
Configuration file, XMPP communication format.
- Protobuf
- A simple overview
Protocol Buffers(Protobuf for short) is a Serialization framework produced by Google. It is independent of development language and platform and has good scalability. Protobuf, like all serialization frameworks, can be used for data storage and communication protocols. Protobuf serialization results are much smaller in size than XML and JSON, which have too much descriptive information, resulting in larger messages; In addition, Portobuf also uses Varint encoding to reduce the space occupied by data. Protobuf serialization and deserialization are much faster than XML and JSON, which convert objects and byte arrays directly. XML and JSON also need to be built into XML or JSON object structures. In a scenario that requires a large amount of data transmission, selecting a Protobuf can significantly reduce the amount of data and network IO, thereby reducing the time consumed by network transmission. Protobuf is a good choice, considering that as a social product, the amount of message data can be very large, and in order to save traffic.
- advantages
- Fast transmission efficiency (smaller size after serialization)
- Support cross-platform multi-language
- Serialization/deserialization is fast
- disadvantages
- Poor readability (binary format)
- Lack of self-description
- Not very convenient to use (it seems that there is no protobuf that supports native C language, most of which are compiled by others)
- Applicable scenario
Scenarios with large data volumes and high transmission efficiency.
Selection of communication framework
The following communication frameworks are commonly used. The advantages and disadvantages of each communication framework and their application scenarios are described.
-
Java Nio
- A simple overview
Java NIO is a buffer-oriented, non-blocking IO interface that is new in Java 1.4. It can be said that it is both New IO and no-blocking IO, but this view is not rigorous. NIO is not only equal to non-blocking IO, there are specific classes in NIO that implement non-blocking IO, but it does not mean that NIO is non-blocking IO. Java NIO consists of the following core parts:
- Buffer the Buffer
- The Channel tunnel
- Selector A multiplexer
Traditional IO operations are stream-oriented, meaning that one or more bytes are read from the stream at a time until complete, without data being cached anywhere. NIO operations are buffer-oriented, and data is read from a Channel into a Buffer Buffer, where it is subsequently processed. The main events of NIO are:
- Read OP_READ ready
- OP_WRITE write ready
- OP_CONNECT Event that the client connects to the server
- OP_ACCEPT The server receives the client connection event
More details can be found in the official documentation.
- advantages
- High concurrency. NIO’s Selector model allows a single thread to monitor multiple input channels, as opposed to traditional BIO(Blocking IO) implementations (a connection consumes a thread resource and thread resources are underutilized). That is, a small number of threads (perhaps a single thread) are enabled to monitor the event pool. Compared with BIO, it saves a lot of CPU resources. BIO establishes one thread for each connection, and the monitoring work is completed by the same thread. NIO divides the monitoring work into a few threads to handle, which of course will not be used for communication, ensuring the utilization of each thread and naturally improving the high concurrency performance.
- disadvantages
- The API is more complex. A good understanding of NIO’s buffers, channels, and selectors is essential to develop robust programs.
- Many additional programming skills are required to use NIO; for example, because NIO involves the Reactor threading model, you must be familiar with multithreading and network programming to write high-quality NIO programs.
- To have high reliability, the workload and difficulty are very large, because the server needs to face frequent client access and disconnection, network intermittent disconnection, half packet read and write, failure cache, network congestion, these will seriously affect our reliability, and the use of native NIO to solve them is quite difficult.
- JDK NIO BUG–epoll null polling, when select returns 0, causes Selector null polling to cause cpu100%.
- Applicable scenario
The server needs to support a large number of long-running connections. Say 10,000 connections or more, and each client doesn’t send too much data too often. For example, a central server in the head office needs to collect transaction information from various cash registers of convenience stores across the country, requiring only a small number of threads to handle and maintain a large number of long-term connections on demand. Jetty, Mina, Netty, ZooKeeper, etc. are implemented based on NIO.
- Mina
- A simple overview
Mina is actually a lot like Netty in that most of the API is the same because it was developed by the same author. However, I feel that Mina is not as mature as Netty, and it is easy to find a solution when problems arise in the process of using Netty. There is not much difference between Mina and Netty in the threading model. The main difference lies in the granularity of task scheduling. In addition, Mina is an early game, and Netty was developed after Mina, with the same author and roughly functional framework, so Netty is a good choice.
- advantages
Much the same as Netty, with some implementation differences.
- disadvantages
- Netty solved a design problem with Mina that tied the kernel to features so tightly that users couldn’t get away from them when they didn’t need them, resulting in performance degradation.
- Netty documentation is clearer and many of Mina’s features are available in Netty.
- Netty’s update cycle is shorter and new versions are released faster.
- The architectures are not very different. Mina lives on Apache while Netty lives on JBoss. The integration with JBoss is very high and Netty has Support for Google Protocal BUf. There is more complete IOC container support (Spring, Guice, JBOSSMC and OSGi).
- Netty is easier to use than Mina, you can customize upstream events or/and Downstream events, and use decoder and encoder to decode and encode the delivered content.
- Netty and Mina handle UDP differently. Netty exposes UDP connectionless. Mina abstracts UDP as a “connection-oriented” protocol, which is difficult for Netty to do.
- Applicable scenario
With Netty.
- Netty
- A simple overview
Netty is an open source Framework based on Java NIO provided by JBOSS. Netty provides asynchronous, non-blocking, event-driven, high-performance, highly reliable, and highly customizable network applications and tools that can be used to develop both servers and clients. Netty is the most popular NIO framework at present. Netty has been widely used in the Internet field, big data distributed computing field, game industry, communication industry, etc. Well-known Elasticsearch, Dubbo, Ali Cloud IoT framework all use Netty.
- advantages
- Elegant design. Uniform API blocking and non-blocking sockets for all transport types; Clear separation of concerns based on flexible and extensible event models; Highly customizable threading model – single thread, one or more thread pools.
- Easy to use. Detailed documentation of Javadoc, user guides, and examples; Without other dependencies, JDK 5 (Netty 3.x) or 6 (Netty 4.x) will suffice.
- High performance. Higher throughput, lower latency, reduced resource consumption, and minimized unnecessary memory replication.
- Security. Full SSL/TLS and StartTLS support
- The community is active. With continuous updates and short version iterations, bugs found can be fixed in time, and more new features will be added.
- High stability. Fixed the infamous epoll empty polling bug in JDK NIO.
- High customization capability. The communication framework can be flexibly extended through ChannelHandler.
- Supports various protocols. Such as TCP, UDP, WebSocket, MQTT, etc., highly encapsulated codecs, easy to use.
- Experienced large-scale commercial application test, quality and reliability have been well verified.
- disadvantages
Not found yet, know can tell the author, thank you.
- Applicable scenario
- Internet industry. In a distributed system, remote service invocation is required between nodes, and high-performance RPC frameworks are essential. Netty, as an asynchronous high-performance communication framework, is often used by these RPC frameworks as a basic communication component.
- The game industry. As a high-performance basic communication component, Netty provides TCP/UDP and HTTP protocol stacks to facilitate customization and development of private protocol stacks and login to servers by accounts. Map servers can easily communicate with each other through Netty.
- In addition to developing instant messaging applications, Netty can also be used to implement HTTP servers.
- Socket.io Socket.IO is also an open source framework for real-time, bidirectional and event-based communication between browsers and servers. I don’t want to go into detail about it.
General Protocol Definition
Mainly to talk aboutProtobuf
File format definition,JSON
It’s a key/value pair. There’s nothing to say.
Let’s first analyze what kind of message format is universal, that is, the unified message format that can be used for single chat, group chat, system messages, etc. This is more important, related to the subsequent expansibility, universality, etc., let’s take a look at the figure:
Corresponding writtenmsg.proto
The code is as follows:
syntax = "proto3"; / / specified protobuf version option java_package = "com. Freddy. Kulaims. Protobuf"; // Specify the package name option JAVA_outer_className = "MessageProtobuf"; Message Msg {Head Head = 1; Body = 2; // message body} message Head {string msgId = 1; Int32 msgType = 2; // Message type string sender = 3; // Sender string receiver = 4; // Receiver int64 timestamp = 5; // Send the timestamp, in milliseconds int32 report = 6; } message Body {string content = 1; Int32 contentType = 2; // Message content type string data = 3; // Extend field, json string stored as key/value}Copy the code
After writing the msg.proto file, follow these steps to generate the MessageProtobufJava class we need to use:
-
In the project SRC /main directory, create a proto folder, the same as SRC /main/ Java.
-
Copy the msg.proto file to the project SRC /main/proto folder.
-
Add to the Dependencies node of the build.gradle file at the project level
The classpath 'com. Google. Protobuf: protobuf - gradle - the plugin: 0.8.12'Copy the code
- in
app
With a magnitude ofbuild.gradle
File, join
apply plugin: 'com.google.protobuf'
Copy the code
- in
app
With a magnitude ofbuild.gradle
Of the fileandroid
Node, join
sourceSets {
main {
java {
srcDir 'src/main/java'
}
proto {
srcDir 'src/main/proto'
}
}
}
Copy the code
- in
app
With a magnitude ofbuild.gradle
Of the filedependencies
Node, join
Implementation 'com. Google. Protobuf: protobuf - Java: 3.8.0'Copy the code
- in
app
With a magnitude ofbuild.gradle
The file root node (that is, andandroid
,dependencies
Node sibling), join
Protobuf {/ / configuration protoc compiler protoc {an artifact = 'com. Google. Protobuf: protoc: 3.8.0'} / / configuration directory generated here, GenerateProtoTasks {all(). Each {task -> task.builtins {remove Java} task.builtins {Java} Task.builtins {Java {}}}}}Copy the code
- Click on the
build->Make Project
, can be generated in the projectBuild/generated/source/proto/debug/Java/java_package proto file specified package name
See the generated ones belowMessageProtobuf.java
Files, files automatically generated, do not need to change:
Note: the above protobuf version is not specified, you can choose your preferred version, but it is strongly recommended to use the same version as the backend version, otherwise there may be compatibility issues.
Many students also use multiple proto files to represent different messages, such as user login messagesuser_login.proto
, chat messageschat.proto
, this is ok, but there will be a lot of proTO files, later maintenance is more troublesome, this is why the need to design a common PROTO file format.
Finally, paste oneJSON
andProtobuf
Serialized byte length comparison graph, two User objects and a timestamp field, you can see that after json serialization, the byte length is140, while the same content after Protobuf serialization has a length of bytes49:
getMessageProtobuf.java
Documentation means that we have completed the first step and are one step closer to developing a commercial-grade IM system, which I will cover in detail in the next articleInterface definition and encapsulation, stay tuned.
Final technology selection
To sum up, in terms of instant messaging, the final technology selection is as follows:
Communication protocol
The communication protocols are TCP and WebSocket, UDP is not considered, and MQTT will be implemented if necessary later.
Transfer protocol
The transport protocol is Protobuf or JSON, which is specified during IM SDK initialization. XML is not considered.
Communication framework
The communication framework is Netty, followed by Java NIO and Mina implementations if required. Socket.IO is not considered.
Write in the last
Before theOpen source a self-used Android IM library, based on Netty+TCP+Protobuf implementation, some students comment, TCP is byte stream oriented, there is no concept of package, which comes of unpacking/sticky package? There is no such thing as TCP unpacking/sticky packets. There is no TCP/IP book on TCP/IP. This is just a misrepresentation, but it has become so popular that the author uses it. The concept of unpacking/sticky packet should only exist at the application layer, TCP does not have sticky packet/unpacking, just no message boundary. This will be explained later in the third article.
Finally finished writing, I found it was too difficult to write the original article, first to consider the way of expression, second to consider whether the layout is beautiful, but also consider whether to meet the needs of everyone, so the procrastination attack again ~ but I will insist on finishing the whole series of articles, improve the project and open source, I hope to help you. The reason is divided into a series of articles to write, on the one hand, because an article is really not clear. On the other hand, I hope that in the process of writing this article, you can give me some comments or suggestions. A person’s energy and level are limited, there are many views may not be correct and perfect, I hope you understand. Welcome to ridicule, welcome to slap, accept criticism.
PS: The newly opened public account can not leave a message, if you have different opinions or suggestions, you can go to the nuggets comment or add to the QQ group: 1015178804, if the group is full, you can also give me a private message on the public account, thank you.Post the official number:
FreddyChen