Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

1. How to ensure message sequence?

The private chat

If a message is transmitted over TCP, if two messages are sent, it is possible to send first and then arrive later.

One solution is:

First, ensure that the order of messages sent by the sender is consistent and correct. For example, the sender maintains a message ID so that it can be sorted by the sender’s message ID.

The order of sender and receiver is then followed by server time as the order of messages. This ensures that the message is sequential.

Group chat

  • Group chat no longer uses the serial number of the sender to ensure the timing, because the sender is not only a point, but also the time is not consistent (the timing of the individual sent, local storage can be implemented, the sent content is already ordered).
  • Serialization can be done using a single point of the server;

This method can be implemented, all group messages are displayed in the same time sequence. The downside is that the service that generates the global increasing sequence number can easily become a system bottleneck.

The further optimization strategy is that the group message does not guarantee the global order, but only guarantees the order of the messages within a group.

In this scenario, the Service layer no longer has to go to a unified back end to get the global serial number. Serialize messages from the same group at the Service layer, ensuring that all group members see messages in the same sequence.

2. How to ensure the reliability of online messages?

Message reliability needs confirmation delivery mechanism, otherwise it is impossible to talk about message reliability.

First, sending A message is A three-way communication, known as A-server-B

Six packets are required for reliable messaging, namely:

  1. Client A sends A message Request package MSG :Request to the Server
  2. The Server sends A reply message MSG :Acknowledge to client A
  3. The Server sends a notification packet MSG :Notify to client B
  4. Client B sends an ACK request package ACK :Rquest to the Server
  5. The Server replies to client B with ACK :Acknowledge
  6. The Server replies the client with ACK :Notify

Therefore, the sending of A message contains the upper half and the lower half respectively, namely, R/A/N packets of Msg and R/A/N packets of ACK. The reliable communication of an application-layer INSTANT messaging message involves six messages, which is the core technology of message delivery in IM system.

3. What if the message is lost? (Timeout retransmission)

We are still based on the design scheme of the above six packets.

  • MSG :Request, MSG :Acknowledge Message lost:
    • The message “send failed” is displayed.
  • MSG :Notify, ACK :Rquest, ACK :Acknowledge, ACK :Notify Packet loss:
    • Timeout retransmission
      • After client A sends MSG :Request and receives MSG :Acknowledge, if client A does not receive ack:Notify within the expected time, client A will try to send MSG :Request again. Client A may send many messages simultaneously. Therefore, client A needs to maintain A local ACK queue and use the timer timeout mechanism to record the messages that do not receive the ACK-N to periodically resend the messages.
      • Once ack:Notify is received, the message is removed from the “wait for ACK queue.”

4. What if the message is repeated due to retransmission?

Retransmission can cause message duplication.

MSG :Notify The retransmission caused by message loss is valid, and the message will not be repeated.

Ack: Indicates the retransmission caused by Notify message loss. Client B has received the ack message but has not received the ACK message from client A.

So how do messages get reduplicated?

In fact, it is very simple. The client generates a deduplicate ID and stores it in the waiting ACK queue. The same message uses the same deduplicate ID to transmit.

5. How to deliver offline messages reliably?

The procedure for sending offline messages is as follows:

  1. User A sends A message to user B.
  2. The server checks the status of user B and finds that user B is in offline state.
  3. The server stores this message in DB as an offline message.
  4. The server returns an ACK acknowledgement from user A that the message was successfully sent (for the sender, once the message is stored in DB, it is considered to have been successfully sent)

The message pull scheme can be pull on demand.

6. Actual business implementation plan

Ensure websocket connection via heartbeat. If disconnected, retry four times every 20 seconds for 80 seconds.

The message is retransmitted twice every 15 seconds.

Reference documentation

  • Instant messaging network