Requirements encountered in business (abstract description) : The ability to implement interval cyclic tasks at different times for different users. For example, 24 hours after the successful registration of users to push relevant SMS to users and other similar requirements.

Use the crontab? It’s too heavy and basically impractical to generate a scheduled task on the server for every user.

Timed polling? I/O is frequent and inefficient

Remember that redis is often used to set the cache time, should have expired event notification, check the documentation, sure enough, there is a related configuration, called “keyspace event notification”. For details, please refer to the official documentation.

Technology stack

redis / nodeJs / koa

Technical difficulties

  • Enable redis keyspace notification (available in versions 2.8.0 and later)
  • Try to use a separate Redis DB for this
  • Use distributed locks based on RedIS to ensure that related events are not re-consumed
  • The information that needs to be used again needs to be represented in the redis cache key
  • The redis cache key uses a service prefix to avoid overwriting the same name
  • This prevents the listening on the NodeJS layer from becoming invalid due to service restart

“talk is cheap, show me the code 🤖”

The core codeconst { saveClient, subClient } = require('./db/redis') // The storage instance and the subscription instance need to be two different instances
const processor = require('./service/task')
const config = require('./config/index')
const innerDistributedLockKey = '&& __ &&' // The eigenvalue of the key of the distributed lock used internally
const innerDistributedLockKeyReg = new RegExp(` ^${innerDistributedLockKey}`)

saveClient.on('ready'.async () => {
  saveClient.config('SET'.'notify-keyspace-events'.'Ex') // The storage instance is set to push key expiration event
  console.log('redis init success')
})

subClient.on('ready', () = > {// All processors can be initialized after the service restarts
  subClient.subscribe(`__keyevent@${config.redis.sub.db}__:expired`) // Subscription instances are responsible for subscribing messages
  subClient.on('message'.async (cahnnel, expiredKey) => {
    // The distributed lock does not listen on the key
    if (expiredKey.match(innerDistributedLockKeyReg)) return
    // Simple distributed lock. The instance that gets the lock consumes the event
    const cackeKey = `${innerDistributedLockKey}-${expiredKey}`
    const lock = await saveClient.set(cackeKey, 2.'ex'.5.'nx') // The usage here enables a simple distributed lock
    if (lock === 'OK') {
      await saveClient.del(cackeKey)
      for (let key in processor) {
        processor[key](expiredKey) // Processor refers to the service logic executed after receiving a key expiration notification, for example, pushing a short message, and then setting a key expiration timer in the processor}}})console.log('subClient init success')})Copy the code
servide/task (processor)
exports.sendMessage = async function sendMessage(expiredKey, subClient) {
  // Only expiration events for related businesses are processed
  if (expiredKey.match(/^send_message/)) {
    const [prefix, userId, type] = expiredKey.split(The '-')
    let user = getUser(userId)
    if (user.phone) {
      push(message) / / pseudo code
      resetRedisKey(expiredKey, ttl) // Set the key to expire after a certain period of time. This logic will be triggered again after the expiration}}}Copy the code

conclusion

  • This function makes use of redis key space notification function to realize the simple task based on users or based on different business scenarios. Since the keyspace event notification function is a CPU intensive operation, it is recommended that a separate DB be used to handle it.
  • The basic usage shown here does not take into account the persistence of scheduled tasks. If redis is restarted during use, all scheduled tasks will be lost. The event will also be lost if the subscription service fails and is not online, or the network problem is not received by the consumer when redis publishes the key failure notification.
  • Redis’ expired event is triggered not when the key expires, but when the key is deleted. Redis will periodically clean expired keys or check if they are expired when accessing the key. Only then will expired keys trigger deletion, so there is a small time gap (personal actual use does not affect the user experience).

Therefore, you need to weigh the usage scenarios of scheduled tasks implemented using redis expiration mechanism.

Thank you for reading. Like friends can follow my public account: Yu Ming Liang Ji, weekly regular update articles oh, including but not limited to technology. I am Yu Ming Liang ji, a love cooking program ape 😜