1, the introduction

In IM systems, especially in enterprise application scenarios, the read and unread status of messages is a strong requirement.

Take Alibaba’s Dingpin as an example. The product positioning of Dingpin is used for business communication, and its “forced read receipt” function makes it impossible for employees to “pretend not to be online” and “pretend not to have received”. What’s more, Dingding.com’s group chat “force read receipt” feature even lets you know who has read a message and who hasn’t (good news for your boss).

▲ Nails in the group chat message has not read function effect

It looks cool, but it’s a long story to use (….) . In fact, the technical implementation is not easy.

So, for the read and unread state:

  • 1) If it is private chat: the message reading state is easier to achieve, and there are no problems in performance and storage;
  • 2) For group chats: Considering storage and processing performance, especially in a cloud environment, how to efficiently deal with the read and unread state of group chats is a topic worth discussing.

There are three aspects to “productivity” :

  • 1) Storage space;
  • 2) Processing speed;
  • 3) Number of bytes transmitted.

This article will discuss the read and unread state from the perspective of the server side, and the difference of ideas on storage space occupation in the specific technical implementation. Limited ability, as personal notes, welcome to exchange.

Learning and communication:

– Im/Push technology development communication group 5:215477170[Recommended]

– Mobile IM Development primer article: Just one Primer: Developing Mobile IM from Scratch

This article has been simultaneously published in the “instant Messaging technology circle” public account, welcome to pay attention to:

Bring this article on the public link is: mp.weixin.qq.com/s/yUkKPOBsd… The original link is: www.52im.net/thread-3054…

2. Content review

Before this article was included, Jack Jiang suggested the original author to share some specific technical points more deeply. However, due to the busy work of the author, some key technical points in this article will be further expanded in the future.

Therefore, this paper can be used as a reference for the basic implementation ideas of the read and unread function in IM chat messages (mainly group chat), but it is not recommended to blindly believe in the conclusion or scheme in this paper, to avoid being misled by some technical indicators that are not specific enough.

3. Related articles

If you want to learn more about the logic of implementing read and unread messages in IM group chats, you can read the article “How to Implement read Receipt for IM Group Chats”. (Highly recommended).

If you are confused about the product pain point of the read and unread functions in IM, you can refer to the design positioning of the read and unread functions in wechat. For details, see “Thoughts on POPULAR IM Features: Why there is no message” Read “function in wechat? .

See appendix for more articles on IM group chat technology.

4. Interactive flow of read and unread states

The system configuration, organization configuration, and group configuration determine whether the recipient needs to be notified of the READ of the IM chat message sent by the sender, or whether the recipient needs to be notified of the read. The following discussion assumes that messages need to be read and unread.

Between the client and server, there are only three commands to read the status, each of which contains a request and a reply.

4.1 Notification Message Read (For Private chat and Group Chat)

When xiao Bao reads one or more messages, a message read notification is sent to the server: “I have read your x+ Y + Z message.”

When the server receives the read notification from Xiao Bao, it needs to complete the following:

  • 1) Store the read state of the message;
  • 2) Return the reply to xiao Bao;
  • 3) Notify the original sender of the message from the read list that the message has been read.

For step “3)” :

  • 1) In the case of private chat, it is easier to understand that it is sent to the other side of private chat;
  • 2) Group chat situations can be quite different: the list of read messages sent by Xiaobao may be sent by your peers. Consider this hypothesis: the group chat messages sent by Zhang SAN, Li Si and Wang Wu are all read by Xiao Bao. Then the message list contained in the read notice sent by Xiao Bao needs to be decomposed by IMS into three read notices (three different message lists), which are respectively sent to Zhang SAN, Li Si and Wang Wu. The contents of the notice are as follows:
    I have read these messages from your excellency

    “.

Here’s a rough logic flow chart:

4.2 Querying the Number of Unread Messages (Private chat and Group Chat)

The sender of the message may need to show whether the message has been read when loading the message list into the chat window.

For a group chat, the message displayed may indicate that n people have not read the message. In this case, you need to query the number of unread messages from the server. Because the client may display multiple messages sent by itself on the UI, you need to query multiple messages at a time.

The unread number is used to express the read status of messages, which unifies the query of private chat and group chat, simplifies the interface between clients and servers, and makes the implementation logic of clients more unified.

Like this:

  • 1) For private chat: if n>0, the message is not read;
  • 2) For group chat: directly display n people have not read, of course, when n equals 0, it means all read.

4.3 Querying the List of Read and Unread Members in a Group Message (Group Chat)

To display the list of read and unread members of a group chat message, the client sends a query to the server.

The general logical flow chart is as follows:

5, several specific read and unread state storage ideas

5.1 Basic Conventions

The reading state of group chat is more complicated than that of private chat, so this paper focuses on the reading state of group chat.

If the number of group members is N, each client immediately sends the READ notification to the IM server. The server needs to store the read status of everyone, including unread members. Because the list of members of the group may change, for example, if a member is added today, the list of recipients of the message sent yesterday is different from that of the message sent today.

That is:

  • 1) For different messages in the same group, the corresponding recipient list may be different.
  • 2) In other words, each message needs to record a complete list of recipients and read people.

For the convenience of discussion, this chapter assumes that there are 640 people in the group.

5.2 Storage Roadmap 1

Each message maintains:

  • 1) Recipient list Receiver_list;
  • 2) Read personnel list read_list.

Concrete is:

  • 1) When IM Server receives a message, build receiver_list with all group members;
  • 2) When the IM Server receives a group member’s read notification of this message, it adds this member to the read_list.

The client gets the data for this message:

  • 1) To obtain the number of unread entries, subtract the number of read_list from the number of receiver_list.
  • 2) To obtain the read and unread personnel list, subtract read_list from receiver_list to obtain the unread personnel list.

So, the storage space for each message is:

640 ids + an indefinite number of read staff ids

5.3 Storage Roadmap 2

Maintenance of each message:

  • 1) Unread personnel list unread_list;
  • 2) Read personnel list read_list.

Concrete is:

  • 1) When IM Server receives a message, unread_list is constructed with all group members;
  • 2) When the IM Server receives a group member’s read notification of this message, it removes the member from unread_list and adds the member to read_list.

The client gets the data for this message:

  • 1) To obtain the number of unread people, directly calculate the number of unread_list;
  • 2) Return unread_list and read_list when you need to get the list of read and unread personnel.

So, the storage space for each message is:

Unread employee ID + Read employee ID: a total of 640 ids

The realization of idea 2 occupies 0.5 to 1.0 times as much space as that of Plan 1. That is, case 2 occupies less space, but each time the client receives a read notification, one more operation is performed than case 1: the unread_list is deleted.

5.4 Storage Idea 3 (My Implementation)

5.4.1) Discuss the deficiencies of Section 5.2 and 5.3:

The two ideas of Section 5.2 and 5.3 can meet functional requirements, but there is a huge storage waste.

There are 640 people in this group. If there are 1024 messages in the chat within the group every day, and the staff ID is calculated in 4-byte storage, the space consumed for the daily message reading status of the group is as follows:

1 24 x (640 x 4 + read number x 4), the value ranges from 2.5MB to 5MB.

1 24 x 640 x 4 is 2.5MB.

This is just a group of read status data generated in a day, if it is running in the cloud platform, only this function consumes space, hehe ~~

As an aside: it would be even more impressive if members were stored as strings instead of 4-byte integers, such as “1123356777”.

5.4.2) How to Reduce storage space:

Considering that group members do not change all the time, and that in most cases the list of group members is relatively stable, and that today’s list may even be the same as last week’s (or even longer) list, it is possible that hundreds or even tens of thousands of messages correspond to the same group member list.

Therefore, the main idea of this paper is introduced:

Consider sharing group membership lists with different messages by storing the read status of messages separately from the group membership list and recording their associations.

Suppose there is a common group member list for every 1024 messages on average. After 1024 messages are sent, the group members change and a new group member list is required.

Then the space occupied by the reading status of the thousand messages is:

Group member list space + 1024 message read status: 640 x 4 + 1024 x Space occupied by each message read status

On the premise of having group member list, how to reduce the space occupied by each message reading state?

It is natural to think of a bit for read people, because a 32-bit integer represents the read state of 32 people. The order of the bits only needs to be the same as the order of the group member list.

When a message has not been read, the read status takes up 0 bytes. The maximum amount of space is taken up when everyone in the group reads, i.e. 640/32 = 20 bytes.

Therefore, after optimization, the space occupied by the reading status of these 1000 messages ranges from 2.5KB to (2.5KB + 1024 * 20B), that is, 2.5KB to 22.5KB, which is greatly reduced compared with the value of thought 1 and 2 in Section 5.2 and 5.3.

As shown below:

Prerequisites for this form:

  • 1) A group of 640 people;
  • 2) The group member list corresponding to the consecutive 1024 messages of the group is stable.

Taking a step back, even if the group member list corresponding to the 1024 messages is unstable and changes 10 times in the middle, there will only be 2.5KB * 10, that is, 25KB more storage space, which is still a great advantage compared with case 1 and case 2.

6. How to improve the processing speed of read and unread states

Xiao Bao sent a message to the company. Let me introduce my new female colleague to you. We checked the message immediately, immediately, instantaneously and in a lightning way.

The process for these messages is the same:

  • 1) These operations can be combined to store and forward in batches;
  • 2) Since storing message reading state is a bit setting process, there is no mutual exclusion problem, even in distributed environment can be safely operated;
  • 3) The member list information corresponding to the message can be temporarily cached in the memory object to reduce query IO and improve efficiency.

Appendix: More technical articles on IM group chat

Fast Fission: Witness the Evolution of wechat’s Powerful Background Architecture from 0 to 1 (I)

How to Ensure “timing” and “consistency” of IM Real-time Messages?

Should I Use “Push” or “pull” to synchronize online status in IM Chat and Group chat?

IM group chat messages are so complex, how to ensure that not lost and not heavy?

Wechat Background Team: Practice Sharing on optimization and Upgrade of wechat Background Asynchronous Message Queue

How to Ensure the Efficiency and real-time performance of Mass Group Message Push in MOBILE TERMINAL IM?

Discussion on Synchronization and Storage of Chat Messages in Modern IM System

Discussion on the Disorder of GROUP Chat messages in IM Instant Messaging

How to Implement the Read Receipt function of IM Group chat Messages?

Should IM group chat messages be saved in one copy (spread read) or in multiple copies (spread write)?

Design practice of a set of highly available, Scalable and Concurrent IM Group chat and Single Chat Architecture

Is it technically possible to pull 1.4 billion Chinese into a wechat group?

IM group chat mechanism, besides circular to send messages, what other ways? How to optimize?

NetEase Cloud Communication Technology Sharing: Practical Summary of The Technical Solution of 10,000 People Chat in IM

Ali Dingding Technology Sharing: The King of Enterprise-level IM — The Excellence of DingDing in back-end Architecture

Discussion on how to Implement the Read and Unread Function of IM Group Chat Messages in Storage Space

>> More similar articles……

(This article is simultaneously published at: www.52im.net/thread-3054…)