Message access requirements for message queues
-
Message queue function: serves as a buffer for components of different processing speeds
-
Message queue requirements
-
Message preservation \
-
Repeat message processing \
-
Message reliability assurance
-
\
List-based message queue solution
-
Message preservation \
- List Natural Guarantee
- Write message lpush
- Read rPOP or BRPOP (block client wait notify, do not consume CPU resources)
-
Duplicate message processing
-
Message queues should be able to provide globally unique IDS (snow Flake) \ for each message
-
The consumer program records the ID number of the message that has been processed
-
Ensure idempotency of consumption
-
-
Message reliability assurance
- BRPOPLPUSH inserts the message into another List backup List
-
Possible problem: Producers send messages quickly and consumers process them slowly, causing messages in the List to pile up and strain Redis memory
-
List does not support multiple consumers. Steam can be used instead
\
A Stream-based message queue solution
-
Streams is a Redis data type designed specifically for message queues (memory-based message queues) \
-
operation
-
XADD: Insert message, ensure order, can automatically generate global unique ID\
-
XREAD: used to read messages, which can read data by ID (set block value) \
-
XREADGROUP: reads messages as a consumer group \
-
The XPENDING command can be used to query for all messages \ that have been read but not confirmed by consumers within each consumer group
-
The XACK command is used to confirm to the message queue that message processing has completed
-
XGROUP Creates consumer groups
-
conclusion
-
Memory message queues use Streams after Redis 5.0
-
Message queue requirements
-
Message sequencing Sequential access to message data
-
Duplicate message processing message data has a globally unique number \
-
Message reliability Ensures that message data is deleted after consumption
-