I. Real-time synchronization

  • Require strong consistency
  • First query slow, if the query can not be saved, then check DB, and then save to the cache
  • When updating the cache, update the database first and then set the cache to expire. (It is recommended not to update the cache content, but to set the cache to expire directly.)

Asynchronous queues

Introduction to the

For high degree of concurrency, asynchronous queue can be used to synchronize data, and message middleware such as Kafka and RicketMQ can be used to process message production and consumption

Kafka advantages

  1. Data on the hard disk, the hard disk for sequential IO stream address, so read efficiency is high

Kafka role

  1. Asynchronous data synchronization
  2. Flow peak clipping: if you send money to 1W people at the same time, the server is under great pressure, you can carry out asynchronous operation, set asynchronous queue, and then timing, when the server is idle, one by one to make money operation
  3. Data is stored on hard disk, not in memory, to ensure that data in the message queue is not lost

Ali’s sync tool Canal

Introduction to the

Canal is implemented by simulating the synchronization mechanism of mysql slave and master and monitoring the log update of DB bitlog to trigger the update of cache. This method can free programmer’s hands and reduce workload, but there are some limitations when using it.

Mysql primary secondary replication principle brief analysis

  • The model has two types of servers: for example, a master server is dedicated to adding, deleting and modifying data, and a slave server (one or more) is dedicated to querying data
  • How to ensure data consistency between two servers?
    1. The master server stores the changes made during the operation ina binary log file. These records are called binary log events. You can view them using the show binlog events command.
    2. The Slave server uses the IO thread to read the binary log file and copy the information to its own relay log file.
    3. The Slave server periodically reads and checks the relay log by starting the SQL thread. If any modification is found, the Slave server immediately reloads the information in the relay log file, executes the modification records in the file, and reports the modification records to the Slave server.

A brief analysis of canal principle

  1. Canal emulated the interaction protocol of the mysql slave, disguised itself as the mysql slave, and sent the dump protocol to the mysql master
  2. Mysql master receives dump request and starts pushing binary log to slave(canal)
  3. Canal parses binary log objects (originally byte streams)

Use UDF to customize functions

In the face of mysql API programming, the use of triggers for cache synchronization, but UDF is mainly c/ C ++ language implementation, learning cost is high.

PS:

  1. Article from a variety of resources collation, such as infringement please inform to delete.
  2. Please indicate the source for reprinting this article