The cache

Why cache

If we want to optimize the latency of an interface, the first thing that comes to mind is caching. Caching reduces response latency for individual requests and increases the number of requests that can be processed per unit of time by reducing data transfer time. Assuming a delay of 10 ms for an interface in a single-threaded process, the TPS is 100 beats/second. It took 3 ms to read the database. If we use Redis to cache the data and reduce the read time to 1 ms, then the interface latency is 8ms and the TPS is 125 beats/SEC. By using caching, the throughput of the interface is increased by 25%.

Why does caching reduce data transfer time? There are two kinds of caches in common use: in-process caches (such as Map) and out-of-process caches (such as Redis).

• When using in-process caching, time is consumed in memory IO.

• When using out-of-process cache, time is consumed in network IO + memory IO.

• When using a database, if the data is not in memory, it has to be loaded from disk. The latency of memory I/O, network I/O, and disk I/O is:

In terms of transmission speed, memory IO > network IO + memory IO > disk IO, that is, in-process cache > out-of-process cache > database.

Type of cache

With caching, read requests are typically executed like this:

• First check to see if there is any data in the cache
• If so, the data is returned directly
• Otherwise, the data is read from the back-end database and written to the cache.

Caches can be divided into read-only caches and read-write caches depending on whether the data in the cache accepts write requests or not.

A read-only cache

Read-only cache: All write requests, including add, delete, and change requests, are sent directly to the back-end database and the data in the cache is deleted.

• advantages
  • The latest data is in the database
  • Data is persisted and will not be lost
• disadvantages
  • When a data is read after modification, it will cause cache miss, so the data needs to be read from the back-end database and written to the buffer. In other words, read-only caches are not suitable for reading or writing data frequently.

Read and write cache

Read/write cache: In addition to read requests, write requests are also sent to the cache to directly modify the data in the cache.

• advantages
  • Writing speed is very fast
• disadvantages
  • The latest data is in the cache, and in the event of a power failure or outage, the data in memory is lost.

The latest data is in the cache and needs to be synchronized to the database. There are two write back strategies: synchronous direct write and asynchronous write back.

• Synchronous direct writing: Write requests are sent to the cache and the database at the same time. Success is returned to the client as long as both return success.
  • advantages
    • Even if the cache is down, the database still has the latest data, ensuring data reliability.
  • disadvantages
    • Increased response latency. A success message is returned to the client only after the database is successfully written.
    • In multithreading, it is easy to cause data inconsistency between cache and database, and it takes a lot of cost to ensure consistency.
• Asynchronous write back: Write requests are only sent to the cache. Write the latest data to the data when the data is obsolete.
  • advantages
    • Writing speed is very fast.
  • disadvantages
    • It is possible to lose data from long ago when an outage occurs. A compromise is to periodically write dirty data to the database, and even if it goes down, only a fraction of the data is lost (depending on the write interval).

Synchronous direct write takes precedence over data reliability, while asynchronous write back takes precedence over write performance.

Applicable scenario

In practice, we can choose different cache types according to the characteristics of the data:

• If you’re reading a lot of data and not writing a lot of data, then read-only caching works better. For example, leaderboard data for an event, live listings.
• If you are reading and writing data frequently or have high response latency requirements, then read and write caching is a better choice.
  • If data reliability is high, synchronous direct write can be adopted in the write back policy. For example, the player’s gold data.
  • If the write back policy has high requirements on data read and write performance, asynchronous write back can be adopted. For example, data for an activity.