“Zefengengchen” wechat technology account is now open, in order to get the first-hand technical articles push, welcome to search attention!

preface

In the previous article, “Multiple Processes: Why Split the Message Service into a Separate Process?” In order to ensure the stability of the connection, we split the application into “main process” and “communication process”, and defined the interface for the two to communicate with each other. Even so, we only realized the client side of the process communication, and to achieve the communication with the other end of the complete chat system role – the server side, we need to rely on the “network communication protocol” to help complete, in this case, we choose the WebSocket protocol.

What is WebSocket?

The word WebSocket, from the word surface can be disintegrated into Web and Socket two words, Socket we are not unfamiliar, it is in the network of different host applications on the process of two-way communication between the endpoint abstraction, is the application program through the network protocol communication interface, A Socket corresponds to a communication end and consists of an IP address and a port. Note that Socket is not a specific protocol, but a logical concept.

So what is the connection between WebSocket and Socket? Can it be understood as the transplant of Socket concept in the Web environment? To answer this question, let’s review the process of Socket programming on the Java platform:

  1. The server creates a ServerSocket instance and binds the local port to listen
  2. The client creates the Socket instance and specifies the IP address and port of the server to connect to
  3. When the client initiates a connection request and the server succeeds in accepting it, an end-to-end TCP connection is established, through which two-way communication is possible. After that, the server continues to be in the listening state and accepts connection requests from other clients.

The above process can also be simplified as:

  1. Server listening
  2. Client request
  3. Connect to confirm

Similarly, the communication between a WebSocket server and a client can be described as:

  • The server creates a WebSocket instance with a valid host and port, then starts and waits for the client to connect
  • The client creates a WebSocket instance and provides it with a URL that represents the server endpoint to which it wants to connect
  • After the client establishes the connection through the HTTP request handshake, it then uses the TCP connection that made the HTTP request for two-way communication.

Originally part of the HTML5 specification, but later moved to a separate standard document to centralize the specification, the WebSocket protocol takes the Socket idea and provides a full-duplex communication mechanism between Web browsers and servers over a single TCP connection. The WebSocket protocol is designed to work well with the existing Web infrastructure. Based on this design principle, the protocol specification defines that the WebSocket protocol handshake process needs to be carried out through THE HTTP protocol, and is designed to work on the same port as HTTP(80) and HTTPS(443), and also supports HTTP proxy and middleware. To ensure full backward compatibility.

Since WebSocket itself is only an application-layer protocol, and in principle can be used by any client that complies with this protocol, we were able to apply it to our Android client.

What is full-duplex communication?

In simple terms, the communication parties (client and server) can send messages to each other at the same time. Why is this important? Because the traditional communication based on HTTP protocol is one-way and can only be initiated by the client, the server cannot actively push information to the client. Once faced with a scenario that has high requirements on real-time data such as instant messaging, when the server has data updates and the client wants to know about them, it can only use the polling method of the client, which can be divided into the following two polling strategies:

  • Short polling

That is, the client periodically sends a request to the server, and the server returns a response immediately after receiving the request and closes the connection. Advantages: simple implementation Disadvantages: 1. Concurrent requests cause great pressure on the server 2. Data may not be updated, resulting in invalid requests. 3. Frequent network requests rapidly consume power and traffic of the client device 4. The time difference between scheduled operations may cause delay in data synchronization. 5. Each request must carry a complete request header

  • Long polling

That is, after receiving the request, the server will block the request if the data is not updated until the data is updated or the connection times out. Advantages: Reduces the number of HTTP requests compared to short polling, saving some resources. Disadvantages: 1. Connection suspension also consumes resources 2. Redundant request headers still exist

The advantage of WebSocket over either of the above solutions is that after the connection is established, all subsequent data is sent in the form of frames. You do not need to re-establish the connection unless one end disconnects the connection. It can therefore be done:

1. Reduce the burden of the server 2. Greatly reduce the unnecessary traffic, power consumption 3. Improve real-time performance and ensure data synchronization between the client and server. 4. Reduce the overhead caused by redundant request headers

Besides WebSocket, what other technologies are available for mobile im?

  • XMPP

Extensible Messaging and Presence Protocol (Extensible Messaging and Presence Protocol) is an XML-based Protocol, which inherits the flexible development in the XML environment. XMPP defines three roles: client, server, and gateway. Communication can occur in both directions between any two of these three. The server is responsible for recording client information, managing connection and routing information. The gateway is responsible for interconnection and interworking with heterogeneous instant messaging systems, including SMS (short message), MSN, ICQ, etc. The basic networking format is for a single client to connect to a single server over TCP/IP and then transfer XML over it. Advantages 1. Super scalability. The extended XMPP can address user requirements by sending extended information. 2. Easy to parse and read. Convenient development and error checking. 3. Open source. In terms of client, server, component, source code library, etc., there are various implementations already. Disadvantages 1. Heavy data load. XMPP performs poorly on mobile devices due to excessive redundant tags and low parsing efficiency.

Application scenario example: Point-to-point negotiation

The company I joined when I just graduated once took over the development of a social platform APP project of online football matchmaking. At that time, in order to improve the communication efficiency during the matchmaking, we considered introducing chat module into the application, and gave priority to the implementation of point-to-point chat function. At that time, the im SDK solution on the market was still not mature, so the team decided to use XMPP+Openfire+Smack as the self-developed technology to build the chat framework. Based on the XMPP protocol and developed in Java, Openfire can be used to build efficient im servers that can support tens of thousands of concurrent users on a single server. Openfire is simple to install and use, and is managed using the Web. Since XMPP is an open protocol, you can log in to the service using any IM client software that supports XMPP. Smack is an open source, easy-to-use XMPP client Java class library that provides an extensible API.

  • MQTT

Message Queuing Telemetry Transport is a lightweight communication protocol based on the publish/subscribe mode and based on THE TCP/IP protocol. The greatest advantage of MQTT is that it provides real-time and reliable messaging services for connecting remote devices with very little code and limited bandwidth. As an im protocol with low overhead and low bandwidth, it is widely used in the Internet of Things, small devices, mobile applications, etc. Features 1. Based on publish/subscribe model. Provide one-to-many message publishing to decouple applications. 2. Low overhead. MQTT clients are lightweight and require minimal resources, and MQTT headers are small to optimize network bandwidth. 3. Reliable messaging. MQTT defines three qualities of service for message publishing to support message reliability: at most once, at least once, and only once. 4. Support for unreliable networks. Designed for limited devices and networks with low bandwidth, high latency, or unreliable.

Application scenarios: Odds update, event live chat room

The main product of my second company is a sports APP that provides simulated guessing and live streaming of events. The core function module of the APP is to provide the latest data of scores and odds of various events. The inefficient HTTP polling scheme mentioned above was initially adopted, with predictable results. After technical reconstruction, MQTT is used to greatly reduce the dependence on the network environment and improve the real-time and reliability of data. When building the live broadcast module in the future, considering the chat room this kind of one-to-many message release scene is also suitable for using MQTT solution, so the original technical solution is used to extend the new chat room module.

  • WebSocket

In contrast, WebSocket features include: 1.** less control overhead. ** The packet header used for protocol control is relatively small when data is exchanged between the server and the client after the connection is created. 2.** Better binary support. **Websocket defines binary frames and can handle binary content more easily than HTTP. 3.** can support extensions. **Websocket defines extensions that allow users to extend protocols and implement some of their own sub-protocols, such as XMPP, MQTT, etc.

Implementation of WebSocket protocol in Android client

WebSocket protocol is very simple to implement, widely used by Android developers of the network request framework – OkHttp WebSocket communication process is encapsulated, provides a concise interface for WebSocket connection establishment, data receiving and receiving, connection alive, connection closed, etc. It allows us to focus on business implementation without paying attention to communication details. It is as simple as implementing the following two steps:

  • Create a WebSocket instance and provide a URL to specify the server address to connect to
  • Provides a WebSocket connection event listener that listens for event callbacks to handle each phase of the connection life cycle

The composition of a WebSocket URL is similar to that of an Http URL in that it consists of protocols, hosts, ports, and paths. The difference is that the protocol name of a WebSocket URL is WS :// and WSS ://. WSS :// indicates a secure WebSocket connection.

First we introduced the OkHttp library dependencies into the project:

Implementation (" com. Squareup. Okhttp3: okhttp: 4.9.0 ")Copy the code

Second, we need to specify the server address to connect to. Here we can use WebSocket’s official server address:

/** WebSocket server address */ private var serverUrl: String = "ws://echo.websocket.org" @Synchronized fun connect() { val request = Request.Builder().url(serverUrl).build() val okHttpClient = OkHttpClient.Builder().callTimeout(20, TimeUnit.SECONDS).build() ... }Copy the code

Next, we call the newWebSocket(request: request, listener: The first is the Request object built above, and the second is the listener for WebSocket connection events. The WebSocket protocol contains four main events:

  • Open: Triggered when a connection is established between the client and server
  • Message: triggered when the server sends data to the client. The data sent can be plain text or binary data
  • Close: triggered when the communication between the server and client ends.
  • Error: triggered when an Error occurs during communication.

Each event is handled by implementing the corresponding callback separately. The listener provided by OkHttp contains the following callbacks:

abstract class WebSocketListener { open fun onOpen(webSocket: WebSocket, response: Response) {} open fun onMessage(webSocket: WebSocket, text: String) {} open fun onMessage(webSocket: WebSocket, bytes: ByteString) {} open fun onClosing(webSocket: WebSocket, code: Int, reason: String) {} open fun onClosed(webSocket: WebSocket, code: Int, reason: String) {} open fun onFailure(webSocket: WebSocket, t: Throwable, response: Response?) {}}Copy the code
object WebSocketConnection : WebSocketListener()
    @Synchronized
    fun connect() {
        ...
        webSocketClient = okHttpClient.newWebSocket(request, this)
    }
    ...
}
Copy the code

The above events are usually triggered passively when the connection state changes. On the other hand, webSockets also provide interfaces for users to explicitly call if they want to actively perform certain operations. The WebSocket protocol contains two main operations:

  • Send () : Sends a message, including text or binary data, to the server
  • Close () : Requests to close the connection.

As you can see, the WebSocket interface provided by OkHttp also provides these two methods:

interface WebSocket { ... fun send(text: String): Boolean fun send(bytes: ByteString): Boolean fun close(code: Int, reason: String?) : Boolean ... }Copy the code

When the onOpen method calls back, the connection is established and data can be transferred. We can then call the Send () method of the WebSocket instance to send a text or binary message. The WebSocket official server will send the data via onMessage(WebSocket: WebSocket, bytes: ByteString) or onMessage(webSocket: webSocket, text: String) is returned to us as is.

How does WebSocket establish a connection?

NewWebSocket (request: request, listener: The WebSocketListener (WebSocketListener) method internally creates a RealWebSocket instance, which is the implementation class of the WebSocket interface. After creating the instance successfully, it calls the Connect (Client: OkHttpClient) method to start an asynchronous connection.

override fun newWebSocket(request: Request, listener: WebSocketListener): WebSocket {
  val webSocket = RealWebSocket(
      taskRunner = TaskRunner.INSTANCE,
      originalRequest = request,
      listener = listener,
      random = Random(),
      pingIntervalMillis = pingIntervalMillis.toLong(),
      extensions = null, // Always null for clients.
      minimumDeflateSize = minWebSocketMessageToCompress
  )
  webSocket.connect(this)
  return webSocket
}
Copy the code

The connection is established by sending an HTTP request to the server. This request contains some additional request headers:

val request = originalRequest.newBuilder()
    .header("Upgrade", "websocket")
    .header("Connection", "Upgrade")
    .header("Sec-WebSocket-Key", key)
    .header("Sec-WebSocket-Version", "13")
    .header("Sec-WebSocket-Extensions", "permessage-deflate")
    .build()
Copy the code

These request headers have the following meanings:

Connection: Upgrade: indicates the protocol to be upgraded

Upgrade: WebSocket: indicates that the webSocket protocol is to be upgraded.

Sec-websocket-version: 13: indicates the WebSocket Version. If the server does not support the version, return a Sec- Websocket-VersionHeader containing the version number supported by the server.

Sec-websocket-key: Is compatible with the sec-Websocket-Accept in the response header of the following server and provides basic protection against malicious or unintentional connections.

When the status code is 101, the server agrees to the client Protocol conversion request and converts it into a Websocket Protocol. This process is called a Websocket Protocol handshake. After the Protocol upgrade is complete, Subsequent data exchanges follow the WebSocket protocol.

We’ve been talking about “shaking hands,” but what is a handshake? In the context of the computer world, a handshake usually refers to the process of ensuring that the server is in sync with its client. Handshake is the basic concept of the WebSocket protocol.

For the sake of visualization, the messages transmitted in the above examples are text. WebSocket also supports binary data transmission, which is accomplished by the “Data Transfer Protocol”, which will be covered in the next article.

conclusion

In order to complete the two-way communication with the server, we choose WebSocket protocol as the network communication protocol, and by comparing the traditional HTTP protocol with other relevant instant communication technologies, we conclude that the following reference standards can be used when choosing the appropriate network communication protocol for the application of mobile devices:

  • Supports full-duplex communication
  • Supports binary data transmission
  • Support extended
  • Cross-language, cross-platform implementation

At the same time, also on the WebSocket protocol in the Android end of the implementation of an example, and WebSocket protocol handshake process for a preliminary glimpse, of course, this is just the first step, the future heartbeat alive, disconnection, message queue and so on each can be a separate subject, will be introduced in the later.

“Zefengengchen” wechat technology account is now open, in order to get the first-hand technical articles push, welcome to search attention!

reference

WebSocket website www.websocket.org/index.html

WebSocket baidu encyclopedia baike.baidu.com/item/WebSoc…

Easy learning WebSocket www.tutorialspoint.com/websockets/…

WebSocket details (a) : a preliminary understanding of WebSocket technology www.52im.net/thread-331-…

Polling and long polling, the long connection, websocket www.cnblogs.com/huchong/p/8…

WEB im: HTTP long link, Long polling (Polling) Details www.52im.net/thread-224-…

WebSockets vs. Long Polling (1) webrtc.org.cn/20190704-we…

MQTT introduction www.runoob.com/w3cnote/mqt…

Android to learn network programming with the Http protocol learning www.kancloud.cn/kancloud/an…