A realtime distributed messaging platform

A brief introduction.

The open source community has a bunch of queue options, intermittently RabbitMQ/Celery/Kue all useful, but more or less, because of the language and maintenance, scalability aspects, are not very satisfactory.

For example, RabbitMQ is difficult to maintain due to language barriers. Celery always have odd questions (I’m not in the right position?) ; Kue relies on Redis so far, it’s pretty convenient, at least my test environment hasn’t had too many problems; HMMM, seems to forget kafka… (Java million black means don’t want to say more)

The reasons for wanting to use NSQ are:

  • Golang development
  • Distributed design
  • The deployment of simple
  • Client Enrichment
  • The HTTP interface is easy to use
  • High stability? (Official website document written, to be verified!)

Use two.

It is said to be simple to deploy. How simple is it? See the following:

# docker-compose.yml version: '3' services: nsqlookupd: image: nsqio/nsq command: /nsqlookupd # ports: # - 4160:4160 # - 4161:4161 nsqd: image: nsqio/nsq command: /nsqd --lookupd-tcp-address=nsqlookupd:4160 depends_on: -nsqlookupd # ports: # -nsqlookupd # ports: # -nsqlookupd # ports: # -nsqlookupd # ports: # -nsqlookupd npm run start depends_on: - nsqlookupd - nsqd nsqadmin: image: nsqio/nsq command: /nsqadmin --lookupd-http-address=nsqlookupd:4161 depends_on: - nsqlookupd ports: - 4171:4171Copy the code

Run:

docker-compose up 
Copy the code

Output:

➔ docker-compose up
Starting nsqdemo_nsqlookupd_1 ...
Starting nsqdemo_nsqlookupd_1 ... done
Starting nsqdemo_nsqadmin_1 ...
Starting nsqdemo_nsqd_1 ...
Starting nsqdemo_nsqd_1
Starting nsqdemo_nsqd_1 ... done
Starting nsqdemo_client_1 ...
Starting nsqdemo_client_1 ... done
Attaching to nsqdemo_nsqlookupd_1, nsqdemo_nsqadmin_1, nsqdemo_nsqd_1, nsqdemo_client_1
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.688524 nsqlookupd v1.0.0-compat (built w/go1.8)
nsqadmin_1    | [nsqadmin] 2017/11/23 09:36:45.519141 nsqadmin v1.0.0-compat (built w/go1.8)
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.690764 TCP: listening on [::]:4160
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:44.690825 HTTP: listening on [::]:4161
nsqadmin_1    | [nsqadmin] 2017/11/23 09:36:45.519747 HTTP: listening on [::]:4171
nsqd_1        | [nsqd] 2017/11/23 09:36:45.626878 nsqd v1.0.0-compat (built w/go1.8)
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639070 TCP: new client(192.168.0.4:34714)
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639093 CLIENT(192.168.0.4:34714): desired protocol magic '  V1'
nsqd_1        | [nsqd] 2017/11/23 09:36:45.627841 ID: 481
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639255 CLIENT(192.168.0.4:34714): IDENTIFY Address:56f210cb639a TCP:4150 HTTP:4151 Version:1.0.0-compat
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.639268 DB: client(192.168.0.4:34714) REGISTER category:client key: subkey:
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.645268 DB: client(192.168.0.4:34714) REGISTER category:topic key:sample_topic subkey:
nsqd_1        | [nsqd] 2017/11/23 09:36:45.628718 TOPIC(sample_topic): created
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:45.648106 DB: client(192.168.0.4:34714) REGISTER category:channel key:sample_topic subkey:sample_topic
nsqd_1        | [nsqd] 2017/11/23 09:36:45.628817 ERROR: no available nsqlookupd to query for channels to pre-create for topic sample_topic
nsqd_1        | [nsqd] 2017/11/23 09:36:45.629965 TOPIC(sample_topic): new channel(sample_topic)
nsqd_1        | [nsqd] 2017/11/23 09:36:45.630113 NSQ: persisting topic/channel metadata to nsqd.dat
nsqd_1        | [nsqd] 2017/11/23 09:36:45.634888 TCP: listening on [::]:4150
nsqd_1        | [nsqd] 2017/11/23 09:36:45.635230 HTTP: listening on [::]:4151
nsqd_1        | [nsqd] 2017/11/23 09:36:45.636008 LOOKUP(nsqlookupd:4160): adding peer
nsqd_1        | [nsqd] 2017/11/23 09:36:45.636032 LOOKUP connecting to nsqlookupd:4160
nsqd_1        | [nsqd] 2017/11/23 09:36:45.639595 LOOKUPD(nsqlookupd:4160): peer info {TCPPort:4160 HTTPPort:4161 Version:1.0.0-compat BroadcastAddress:b10017538479}
nsqd_1        | [nsqd] 2017/11/23 09:36:45.642355 LOOKUPD(nsqlookupd:4160): topic REGISTER sample_topic
nsqd_1        | [nsqd] 2017/11/23 09:36:45.645742 LOOKUPD(nsqlookupd:4160): channel REGISTER sample_topic sample_topic
nsqd_1        | [nsqd] 2017/11/23 09:36:45.648649 LOOKUPD(nsqlookupd:4160): REGISTER sample_topic sample_topic
client_1      | npm info it worked if it ends with ok
client_1      | npm info using [email protected]
client_1      | npm info using [email protected]
client_1      | npm info lifecycle [email protected]~prestart: [email protected]
client_1      | npm info lifecycle [email protected]~start: [email protected]
client_1      |
client_1      | > [email protected] start /usr/src/app
client_1      | > node index.js
client_1      |
nsqd_1        | [nsqd] 2017/11/23 09:36:47.295258 TCP: new client(192.168.0.5:47104)
nsqd_1        | [nsqd] 2017/11/23 09:36:47.312326 CLIENT(192.168.0.5:47104): desired protocol magic '  V2'
nsqd_1        | [nsqd] 2017/11/23 09:36:47.318512 [192.168.0.5:47104] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}
nsqlookupd_1  | [nsqlookupd] 2017/11/23 09:36:47.327151 200 GET /lookup?topic=sample_topic (192.168.0.5:51278) 28.186µs
client_1      | Message sent successfully
nsqd_1        | [nsqd] 2017/11/23 09:36:47.333635 ERROR: [192.168.0.5:47104] - E_INVALID cannot CLS in current state
nsqd_1        | [nsqd] 2017/11/23 09:36:47.334235 PROTOCOL(V2): [192.168.0.5:47104] exiting ioloop
nsqd_1        | [nsqd] 2017/11/23 09:36:47.334762 ERROR: client(192.168.0.5:47104) - E_INVALID cannot CLS in current state
nsqd_1        | [nsqd] 2017/11/23 09:36:47.334901 PROTOCOL(V2): [192.168.0.5:47104] exiting messagePump
client_1      | Writer closed
nsqd_1        | [nsqd] 2017/11/23 09:36:47.353106 TCP: new client(192.168.0.5:47108)
nsqd_1        | [nsqd] 2017/11/23 09:36:47.353424 CLIENT(192.168.0.5:47108): desired protocol magic '  V2'
nsqd_1        | [nsqd] 2017/11/23 09:36:47.354359 [192.168.0.5:47108] IDENTIFY: {ClientID:0b1264dbb8b4 Hostname:0b1264dbb8b4 HeartbeatInterval:30000 OutputBufferSize:0 OutputBufferTimeout:0 FeatureNegotiation:true TLSv1:false Deflate:false DeflateLevel:6 Snappy:false SampleRate:0 UserAgent:nsqjs/0.10.0 MsgTimeout:0}
client_1      | Received message [08e1925aee1e1000]: it really tied the room together
client_1      | Received message [08e1925aee1e1001]: This message gonna arrive 1 sec later.
client_1      | Received message [08e1925aee5e1000]: Uh, excuse me. Mark it zero. Next frame.
client_1      | Received message [08e1925aee5e1001]: Smokey, this is not 'Nam. This is bowling. There are rules.
client_1      | Received message [08e1925aee5e1002]: Wu?
Copy the code

Management interface:


Reference 3.

  • A realtime distributed messaging platform
  • Demo code: Thonatos/NSQ-Demo
  • node client: dudleycarr/nsqjs