preface

May friend most of the work life of have more than three years or four years, five years do not know whether there is a sense of crisis, we wrote so many demand code no lines 20 w has a 10 w, but I went out looking for work is not written by pass away is the interview by pass, you will find a lot of you just know but couldn’t come up with an answer. This time you just know go remedial knowledge point, in fact this kind of practice is not very friendly to oneself development.

During the epidemic last year, when everyone was afraid of job-hopping, I chose job-hopping without hesitation and entered the bat first-line factory. Recently, I had a chat with colleagues from my former employer about the technology stack and daily life. Said that interviewed a lot of two or three years of experience the developer base is too weak and so on.

Therefore, I started writing with my former friends at the end of April. I basically carried out my writing mode in the way of explaining to small white people. During this period, some friends asked me to write advanced 😭 😭 😭, but I did not dare to do so in front of such big men. Or adhere to the redis from 0 to 1 explain the road, hope to get the support of small partners.

  • I finally figured out the string application scenario for redis data structures
  • Interview Series -2 Redis List scenario analysis practices
  • Interview Series 3 Practice in traffic limiting Scenarios
  • Interview Series -4 Hash Application Scenario Analysis Practice
  • My wife asked me what is synchronous, asynchronous, blocking, non-blocking
  • Ops made fun of me. You didn’t know that?

Most of them are based on their previous interviews and colleagues to communicate, there are wrong places also hope friends more correct.

So open our new round of interview knowledge tour…..

The body of the

Kafka, RabbitMQ, ActiveMQ and RocketMQ are all used in redis. I’ve probably used three of these, but the implementation principle and internal usage are pretty much the same. Why redis? Because light weight, direct use, and the above several suitable for large data volume, high requirements for data accuracy of the scene, as a third-party component in small companies considering the cost of manpower is not too good, there are more risks.

Why publish subscriptions

In fact, in theory, our previous list scenario could implement publish and subscribe using a double-entailed linked list. However, there are two limitations of the publish and subscribe function implemented through a linked list:

  • 1. Message queues based on linked lists cannot support one-to-many message distribution.
  • 2. If the rate at which producers generate messages is much higher than the rate at which consumers consume messages, it may result in unconsumed messages taking up a large amount of memory (requiring sufficient consumption processes to be opened).

Let me draw two pictures for comparison. You can see the difference at a glance:

As you can see from the figure above, a normal message queue can only be consumed by one or more consumers, but messages cannot be distributed to other consumers. Redis subscription publishing: The producer produces the message and distributes the message to the consumers who subscribe to the channel through the channel. This can reduce the accumulation of queue data and lead to memory explosion.

So in order to solve these two limitations, Redis chose to implement publish and subscribe mode through other commands.

Redis subscribes to the basic command for publishing

Psubscribe: pSUBSCRIBE pattern [pattern...] Subscribe to one or more channels that fit a given pattern; Time complexity O(n), where n is the number of subscribed schemas. Note: Each mode is matched with *; For example, mumu* matches all channels starting with mumu: mumu. Juejin, mumu. Zhihu, mumu. CSDN publist directive: publish Channel Message Publishes message to the specified channel channel; The time complexity is O(n+m), where n is the number of subscribers for the channel and M is the number of clients using Subscribed patterns. Note: The result set returns the number of subscribers that received message and 0 for none. Pubsub channels [argument [argument...]] Check the state of the subscription and publishing system. Time complexity O(n), where n is the number of active channels (for short channels and patterns, the complexity of pattern matching is regarded as a constant). Note: Listing the current active channels (i.e. those with at least one subscriber, subscription mode clients are not counted) returns a list of active channels. Punsubscribe [pattern [pattern...]] unsubscribe all channels of a given pattern; Time complexity O(n+m), where N is the number of patterns subscribed by clients and m is the number of patterns subscribed by all clients in the system. Note: if pattern is not specified, all subscribed patterns will be unsubscribed; Subscribe: subscribe channel [channel... Subscribe to a given channel or channels of information; Time complexity O(n), where n is the number of channels subscribed. Unsubscribe: unsubscribe channel [channel...] To unsubscribe from a given channel; Time complexity O(n), where n is the number of channels subscribed. Note: If channel unsubscribe is not specified, all channels are unsubscribe by default. Otherwise, unsubscribe the specified channel. All channels subscribed to the BSCRIBE command are unsubscribed. In this case, the command returns a message informing the client of all unsubscribed channels.Copy the code

So publishing and subscription in Redis are also divided into two types, one is implemented based on channel, the other is implemented based on pattern.

Channel based implementation of the explanation

  • subscribe channe1 channel2 channel3 … Subscribe to one or more channels
  • unsubscribe channe1 channel2 channel3 … : Unsubscribe from subscribed channel (close client terminal does not work, you need to command unsubscribe)
  • Publish CHANne1 message: sends a message to the specified channel
  • Pubsub numsub channel1 channel2: Views the number of subscriptions to the specified channel

A bad pen is better than a good memory.

127.0.0.1:6379> SUBSCRIBE MUMu_1 MUMu_2 Reading messages... (press ctrl-c to quit) 1) "subscribe" -- return type: 2) "MUMu_1" -- the name of the subscribed channel 3) (integer) 1 -- the number of subscribed channels 1) "subscribe" 2) "mumu_2" 3) (integer) 2 1) "message" -- Type of return value: Message 2) "MUMu_1" -- source (sent from that channel) 3) "\xe6\x88\x91\xe6\x98\xaf\xe9\x98\ XBF \xe6\ XB2 \x90\xe5\x95\x8a" -- message contentCopy the code

127.0.0.1:6379> PUBSUB numsub mumu_1 mumu_2 1) "mumu_1" -- channel name 2) (integer) 1 -- Number of clients subscribed to the channel 3) "Mumu_2" 4) (INTEGER) 1 127.0.0.1:6379> Pubsub channels 1) "MUMu_2" 2) "mumu_1Copy the code

127.0.0.1:6379> UNSUBSCRIBE MUMu_1 1) "UNSUBSCRIBE" -- type of return value: show UNSUBSCRIBE success 2) "mumu_1" -- name of UNSUBSCRIBE channel 3) (integer) 0Copy the code

Let’s take a look at the implementation principle based on channel:

Redis-5.0.7 / SRC /server.h About 1239 lines.

struct redisServer { /* General */ pid_t pid; Dict *pubsub_channels; 😜 😜 😜 dict *pubsub_channels; /* Map channels to list of subscribed clients */ }Copy the code

Pubsub_channels defines an attribute that is a dictionary type that stores client and channel information. The key value stores the channel name and the value is a linked list that stores client ids.

Channel subscription: check whether the field exists inside the channel subscription; If not, create a dictionary for the current channel and create a linked list to store the client ID; Otherwise, insert the client ID directly into the linked list.

Unsubscribe channel: delete the client ID from the corresponding linked list when unsubscribe channel; If the list is already empty after deletion, the channel will be removed from the dictionary.

Explain the implementation based on patterns

  • psubscribe pattern1 pattern2 pattern3 … : Subscribes to one or more channels that conform to a given pattern, each pattern being matched by *
  • punsubscribe pattern1 pattern2 pattern3 … : Unsubscribe mode (closing the client terminal is useless, you need to command unsubscribe)
  • Pubsub Numpat Pattern1 returns the number of subscribed patterns, not the number of clients that subscribed to the pattern, but the sum of all the patterns that the client subscribed to. Time complexity O(1),

When it comes to talking about it, hurry up and practice, seeing is believing:

127.0.0.1:6379> Subscribe mumu.* Reading messages... (press ctrl-c to quit) 1) "psubscribe" -- return type: 2) "mumu.*" -- subscribed schema 3) (integer) 1 -- subscribed schema number 1) "pmessage" -- returned value type: Message 2) "mumu.*" -- the pattern of information matching 3) "mumu. List "-- the target channel of the message itself 4)" I am a mumu" -- the content of the messageCopy the code

127.0.0.1:6379> PUNSUBSCRIBE mumu.* 1) "PUNSUBSCRIBE" -- return type: 2) "mumu.*" -- unsubscribed schemas 3) (integer) 1 -- Number of unsubscribed schemasCopy the code

Let’s look at the schema-based implementation principle:

Redis-5.0.7 / SRC /server.h About 1240 lines.

struct redisServer { /* General */ pid_t pid; 😄 😄 😄 list *pubsub_patterns; list *pubsub_patterns; /* A list of pubsub_patterns */ / typedeft struct pubsubPattern {client *client; -- Subscription client robj *pattern; -- Subscribed patterns} pubsubPattern;Copy the code

Pattern subscription: Add a pubsub_PATTERN data structure to the end of the linked list and save the client ID.

Unsubscribe schema: Remove the schema subscription that needs to be unsubscribe from the current linked list pubsub_Patterns structure.

From the above some practical results and combined with the graph is not redis publishing subscription further understanding?

So what can we do with redis publishing subscriptions?

Pub/SUB: A subscriber subscribes to a channel. Publisher is responsible for sending binary string messages to the channel, which, when received, pushes them to subscribers.

  • In e-commerce, users send messages to the designated channel after placing an order, and the downstream business subscribes to the payment result. This channel processes its own business logic
  • Fan following function
  • The article push
  • Etc., etc.

Practice coding

The consumer subscribes to subscribe.php
<? PHP /** * Created by * Date: 2021/05/04 * Time: 16:00 * QQ: [email protected] */ // set PHP script execution time set_time_limit(0); $channel_names = ['mumu_test1', 'mumu_test2', 'mumu_test3']; $cur_time = time(); Try {// instantiate redis $redis = new redis (); $redis->pconnect('127.0.0.1', 6379); //echo "Server is running: " . $redis->ping(); While (true) {$redis instance of redis $channel_name channel name $MSG Message body generated by producer $redis->subscribe($channel_names, function ($redis, $channel_name, $msg) { switch ($channel_name) { case 'mumu_test1': echo "channel:".$channel_name.",message:".$msg."\n"; break; case 'mumu_test2': break; case 'mumu_test3': break; } if (! Echo "channel:".$channel_name.",message: not appoint channel name"."\n"; sleep(1); }}); If (time() - $cur_time > 10*60){$redis -> close(); break; } } } catch (Exception $e) { echo $e->getMessage(); }Copy the code
The producer sends the message publish.php
<? PHP /** * Created by * Date: 2021/05/04 * Time: 16:00 * QQ: [email protected] */ / $channel_names = ['mumu_test1', 'mumu_test2', 'mumu_test3', 'mumu_test4']; $channel_name = $channel_names (rand (0, 3)); $redis = new redis (); $redis = new redis (); $redis->connect('127.0.0.1', 6379); for ($i = 0; $i < 10; $i++) { $data = array('key' => 'key' . ($i+1), 'msg' => 'I am li a mu ! '); $ret = $redis->publish($channel_name, json_encode($data)); print_r($ret); } } catch (Exception $e) { echo $e->getMessage(); }Copy the code

🐧 Run the result set

/usr/local/opt/[email protected]/bin/ PHP subscribe.php; The result set: ➜ publish-subscribe git:(master) qualify /usr/local/opt/[email protected]/bin/ PHP subscribe.php channel:mumu_test1,message:{"key":"key1","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key2","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key3","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key4","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key5","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key6","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key7","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key8","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key9","msg":"I am li a mu !" } channel:mumu_test1,message:{"key":"key10","msg":"I am li a mu !" /usr/local/opt/[email protected]/bin/ PHP publish.php; /usr/local/opt/[email protected]/bin/ PHP publish.php; The result set: ➜ publish-subscribe git:(master) qualify /usr/local/opt/[email protected]/bin/ PHP publish. PHP 1111111111 // Mumu_test4 causes the result set returned to be 0, indicating that there are no subscribed channels. ➜ publish-subscribe git:(master) qualify /usr/local/opt/[email protected]/bin/ PHP publish.php 0000000000Copy the code

🐮 Precautions

1. The consumer of the subscription needs to keep executing, blocking the retrieval message and unsubscribing if it breaks.

A channel receives messages that publish and does not store them. If a channel is not subscribed, the message is discarded.

When a producer generates a message, it simply drops the message into the channel.

🙈 and of course these commands can be played with

$redis->pubsub('channels'); $redis->pubsub('channels', 'pattern '); $redis->pubsub('numsub', ['channel1', 'channel2']); $redis->pubsub('numpat'); $redis->unsubscribe(['channel1', 'channel2']); $redis->punsubscribe(['pattern1', 'pattern2']); // The client unsubscribes all specified modesCopy the code

Small partners local practice operation ~~~, thousands of see as write again.

The pros and cons of redis publishing subscriptions

Friends, from the above practice, if there is no corresponding channel or consumer for the message produced by PubSub, the message will be discarded, and the state of 0 will be returned after direct delivery failure. If one of the consumers is suspended for a period of time due to sudden network fluctuations during consumption in our actual production environment, then when it is reconnected, the messages generated in the intervening period will not exist. That is, Redis itself does not store message body information.

So when we have a small number of production environments and want to save costs, redis’s publish and subscribe function may be more suitable for our company. Lightweight and easy to use with Consul + Supervisor + SWool resident memory, open multi-process consumption (message queue can also be used).

🐣 summary

Wow, wow, lucky to see my friends here, I am so convinced by you. I spent two days to think, draw and conceive the written article. You actually also saw here, a mu heart thief happy; A mu really admire partners, thief stick 👍, thief has perseverance; At the same time, I can tolerate the shortcomings of AMu.

This paper mainly through sorting out the actual operation of PubSub instructions, and then combined with the underlying source code analysis of the storage structure between them; Then through the actual client operation, to explain the specific meaning of the returned parameter; Finally, write code to run the demo through practice. At the same time, the advantages and disadvantages of PubSub are listed to help people have a better choice in practical work. Finally, a good memory is better than much hands-on practice. Only practice can know its essence.

Finally, please pay attention to my personal public account “I am AMu”. I will update back-end knowledge points and study notes irregularly. Also welcome to contact me directly on the public account, private message or email, we can learn together, progress together.

Ok, I am a mu, a worker who does not want to be eliminated at the age of 30 ⛽️ ⛽️ college. Writing is not easy to feel “A mu” wrote a bit of material words: 👍 pay attention to, 💖 share, we will see you next time.