Goim Article Series (6 articles in total):

  • Goim architecture and customization
  • From goIM customization, talk about golang interface decoupling and gRPC
  • Basic concepts and applications of Bilibili/Discovery (Eureka) in GOIM
  • Goim data Flow Data flow
  • Goim business Integration (Sharing session summary and QA)
  • What is watchOps listening for in Goim?

[Brief description] Goim. IO is a very successful IM (Instance Message) instant messaging platform. This article summarizes the goIM exchange and sharing meeting held by go Night Reading organization

Goim communication sharing video, the first video conference, a bunch of questions, please bear with me 🙁

The video is at YouTube www.youtube.com/watch?v=bFn…

1. System integration and expansion of GOIM

Here’s a possible way to do goIM custom extensions, or optimizations

  1. AAA/LB and User Management Sub-system (UMS) are added to support user registration, activation, and permission management. In particular, AAA can send the following data to users after successful login authentication
    1. Goim Json token that specifies which room the current user can enter and which room to receive messages from
    2. Returns the Comet address of GoIM (entering room to receive IM messages) and the Logic address (sending IM messages)
  2. Extended session Server session management to add user online and offline status to support offline messages
  3. Extension on Logic or Comet to add message storage and related management, retention of server group messaging (broadcast or multicast), support for chatbots, add instant message storage or processing interface, such as offline message storage, users online to retrieve offline messages (background triggered send)
  4. Add room management, open IM room chat room, switch chat room, chat room personnel management/group master (administrator, etc.)
  5. Custom extensions on Comet add message sending on the client side to send/receive instant messages streaming in both directions

2. Authentication integration

2.1 Before Interaction with GOIM (Login Authentication and LB Traffic Diversion)

Goim clients, or GOIM terminals, should interact with AAA or similar nes before interacting with GOIM:

  1. Log in using the user name, password, or related information to obtain the current user ID and related authentication information. For details, see the token in the next section
  2. AAA either returns the necessary routing data to the GOIM terminal, such as which Comet (to receive IM messages) and which logic (to send IM messages) goIM should access, or gives the GoIM terminal a Discovery address, Go to discovery to get comet/logic ————-

2.2 authentication token

The following code is taken from /examples/goim-web/public/client.js

function auth() {
     var token = '{" mid ": 123," room_id ":" live: / / 1000 ", "platform" : "web", "accepts" : [1000100, 1100] 2}'
     
     # see/API/comet/GRPC/prototol. Go
     # see func (p *Proto) WriteTo(b * xbytes.writer)var headerBuf = new ArrayBuffer(rawHeaderLen); var headerView = new DataView(headerBuf, 0); var bodyBuf = textEncoder.encode(token); headerView.setInt32(packetOffset, rawHeaderLen + bodyBuf.byteLength); headerView.setInt16(headerOffset, rawHeaderLen); headerView.setInt16(verOffset, 1); headerView.setInt32(opOffset, 7); headerView.setInt32(seqOffset, 1); ws.send(mergeArrayBuffer(headerBuf, bodyBuf)); . }Copy the code

The token is reported when goIM connects to websocket as follows

{
  "mid": 123,                 ## memberID is the current goim memberID
  "room_id": "live://1000".## Goim member room number, chat room number, group number
  "platform": "web".## goim terminal type
  "accepts": [                Goim specifies the room IM messages that terminal users can acceptIn 1000,## goim user switch room, which is handled here1001, 1002]}Copy the code

2.3 Authentication integration

There are two integrated processes for authentication

  1. Before goim can connect to Comet, the user should obtain the user information to construct goIM’s authentication token, which is mid/Room_id/accepts the data mentioned in the previous section
  2. /internal/logic/conn.go func (l * logic) Connect(c context.Context, server, cookie string, Token []byte) handles the verification of connection authentication, such as checking whether mid exists, whether the corresponding Room_id/accepts permission to enter, etc. If the verification passes, session data is generated and stored in Redis. Otherwise return authentication failure (in Comet, authentication failure is handled by closing goim’s long connection to Comet)

3. Send and process IM messages

Im message processing, including the following

  1. Message storage
  2. Interactive processing of messages, e.g., chatbots, sensitive message filtering (social features)
  3. Processing of offline messages
  4. QOS for sending messages
  5. Goim clients send upstream via Comet

4. QA

  1. What is a comet

Comet [Measure] : Based on HTTP long connection “server push” technology, is a new Web application architecture

In the applications developed based on this architecture, the server will actively push data to the client program in an asynchronous manner, without the need for the client to make an explicit request. The Comet architecture is ideal for event-driven Web applications, as well as applications that require high levels of interactivity and real time, such as stock market analysis, chat rooms, and web-based online games.

Server Push is one of the hottest buzzwords in Web technology these days. It goes by the nickname Comet. It is another popular Web technology after AJAX. The recent popularity of server push technology is closely related to AJAX. ———— Baidu Encyclopedia

In my telecom-related solutions, this term is used to replace adapter/gateway, the more common term used to represent the connection point between back-end services and terminals. It is usually deployed behind proxy or Load Balance and provides interface services for terminals and clients

  1. Kafka/rabbitMQ/activeMQ/PALsar/NSQ/NATS

In GOim, the default MQ is kafka, a Java implementation that has many successful large-scale deployment and long term operational business cases for a heavyweight MQ message queue

The new thing is PalSAR (Java implementation). Look at the technical whitepaper. It’s very good

ActiveMQ has been evaluated and compared, and is not recommended in my evaluation report

RabbitMQ has been used, MQ message transfer is not particularly efficient and stable

NSQ C ++ implementation, relatively low level, high efficiency but a lot of MQ related business code to write their own, NSQ author also used pure C rewrite NSQ, called nano

Nats is a go implementation, lightweight, extremely efficient, and lacks message persistence. It’s my personal favorite, and it’s easy to deploy

The choice of MQ, IN my opinion, is more about operations support than business functionality, especially for operations and tuning in very large scale deployments. Integration of MQ with the business is a minor consideration. So, my advice is to choose the MQ that your operations/technical team is most familiar with and the one that has the most commercial success stories and is ecologically active.

In Goim, as in Kafka for Site B, it should be balanced.

As for which MQ to use in your business, the advice is to let operations choose first, as it is more important to have long-term stability with operations support.

  1. Can THE gRPC in GOIM be replaced

Yes, you can change it. For GO, it would be good to switch to RPCX, which can be used for low-level communication protocols such as TCP/KCP or even UDP, protobuf/flatbuffers/JSON or custom RPC data serialization/deserialization……..

But in general, replacing gRPC is not recommended.

It is easy to replace gRPC with RPC or other communication middleware. Development teams are advised to use RPC middleware that they are most familiar with and good at. If you don’t have the middleware you’re best at, keep gRPC.

More mainstream RPC framework, gRPC is not the best choice, for example, gRPC lacks its own service registration/discovery, load balancing and scheduling these commercial RPC necessary parts, but win in Google endorsement, the community is still active, since 2018 to now, progress is still

  1. 58. Is it possible to use GOIM as a communication server or as a communication component in games

Yes, in theory it can

But it’s not recommended

Technology selection is complex, but simple: consider the current/future business scenarios (including the various environments in which the business operates……) And the combined costs of operations/operations/R&D to support these businesses

While GoIM has long connections and a decent deployment architecture, it is still a technical model accumulated in the messaging/barrage business scenario, rather than an optimal choice in the business/technical accumulation of game development, or communication middleware

When it comes to game development, I’m a white elephant. But there are two things that I think are important in game development:

  1. Session state data in various games, as well as the health that characters share or even interact with each other, and the location data of characters in scenes (such as maps)……… This data needs to be synchronized and processed quickly in the game logic, which is key
  2. Efficient communication between the game terminal and the server, such as the famous 16ms principle, communication is slow, the game will stall

The first is the distributed architecture, the second is the custom serialization/deserialization of data in communication interaction (in goIM’s case, danmarek messages), and the combined transmission of multiple messages, which is very clever and effective. _

_

_

5. Conclusion

Goim series of short articles, this one is the end.

Thanks to friends who like open source, I wrote these articles in my spare time during the 360-degree fancy inquiry and exchange over the past month.

In the evening of May 30, 2019, I had a video conference with more than 70 friends about GoIM and surrounding technologies for one and a half hours, which was also an interesting experience: IN the video, I looked like a customer service man facing the screen, ha.

Once again, thank youwww.bilibili.comOpen source &Mao JianBig God, and many predecessors of the open source community, friends

About me

Online name Tsingson (Sanmingzhi, known as Uncle 3)

Original Ustarcom IPTV/OTT Business Unit Broadcast control product line Technical architecture Wet/solution engineering Wet role (8 years), freelancer,

Like music (harmonica, one of the main planners of the 3rd / 4th / 5th Guangdong International Harmonica Carnival), photography and cross-country,

Golang language (postgres + Golang for commercial projects)

_

_

Tsingson written by Xiao Luo Harmonica Music Center, Shenzhen, China, 2019/05/30, last updated on 2020/01/06