1. The background

The company is working on a social project, and the audio and video technology is a third-party technology. Related functions such as broadcasting, entering and leaving rooms and pushing need to be developed by ourselves, so we need to build a long-connection server by ourselves. Therefore, in terms of technology selection, SpringBoot + Netty is used to build long-connection services in order to ensure high concurrent performance and long-connection performance of services. SpringBoot and Netty frameworks are not the focus of this paper. This paper focuses on Netty cluster construction and message forwarding function.

2. Technical options & implementation

2.1 About Netty Long Connection

Netty is a very good NIO asynchronous event-driven framework, on the basis of JDK NIO, packaging and expand packaging, making NIO API easy to use, many frameworks are also used (such as Dubbo). Netty, Netty can be used as a long-connection service, using THE WS protocol, can build a high-performance long-connection server, it is said that a single machine can support about 10,000 connections.

Those familiar with Netty or NIO know that the underlying data is transmitted through a network connected to a Channel. For stand-alone Netty, channels are all connected to the same server, and the communication between channels can be directly obtained and forwarded according to the bound user information. However, for a Netty cluster, each client may be connected to a different server. In this case, when a Channel communicates with each other, it needs to determine whether the Channel is on the current node and forwards the message.

2.2 Technical Options

There are many technical frameworks to realize message forwarding, such as ZK, MQ, Redis, etc. Since the current project has built its own MQ and Redis services, considering the cost, it only selects MQ and Redis at present.

As mentioned above, after the Netty cluster is established, a Channel needs to determine whether the target user’s Channel is on the node after receiving the message. If not, it needs to forward the message to the target node and write the message to the Channel. So MQ’s broadcast mechanism and Redis’s publish and subscribe can do the job.

The comparison is as follows:

  • MQ: dedicated to message queuing, high reliability, but heavyweight, asynchronous, not guaranteed real-time.

  • Redis: lightweight, low latency, high concurrency, low reliability.

Both are used in the project. Considering the timeliness of live broadcast news, Redis is chosen to publish and subscribe, which is relatively simple to integrate.

2.3 Implement the architecture diagram

① When the service starts, register its node information with Redis and specify the listening channel.

② After the client successfully connects to the Netty Server, it binds the node that the user joins to the Redis cache for subsequent information forwarding and the information is forwarded to the specified channel.

③ During Channel forwarding, if the current user is in the node, it will be directly forwarded; if not, the registered node information will be obtained from Redis and the message will be published to the designated Channel node.

3. Code implementation

Binds the current node, the service starts and listens for the specified channel, specifying a Listener.

Subscription node generator native IP + service port

Listener implementation, listening to the Channel received messages, processing and sent to the corresponding Channel

Message forwarding Sender

4. To summarize

Through the application of Redis to publish and subscribe, Netty Server multiple clusters, when adding multiple nodes, the new node will automatically register to monitor the corresponding channel, message forwarding will also automatically delivered to the newly registered node channel, scalability.

At present, there are many new advanced features in Redis 5, which the author is also learning slowly and can be applied to the project business scenarios.

Source: www.jianshu.com/p/902a8fd42…

My wechat official account: Java Architect advanced programming focus on sharing Java technology dry goods, looking forward to your attention!