• Advanced Python: How To Implement Caching In Python applications
  • Farhad Malik
  • The Nuggets translation Project
  • Permanent link to this article: github.com/xitu/gold-m…
  • Translator: jaredliw
  • Proofreader: KimYangOfCat, Greycodee

Python Advanced: How to implement caching in Python applications

Caching is an important concept for every Python programmer to understand.

In short, caching is the use of programming techniques to store data in temporary locations rather than retrieving it from the source every time.

In addition, caching improves application performance because it is always faster to access data from temporary locations than to retrieve data from sources (databases, servers, and so on).

This article focuses on how caching works in Python.

This is an advanced topic for Python developers. If you are using Python or plan to use it, this article is for you.

I highly recommend reading this article if you want to learn the Python programming language from beginning to end.

The gist

In this article, I explain what caching is, why we need it, and how to implement it in Python.

Caching is used when we want to improve the performance of our applications.

I will outline the following three key points:

  1. What is caching and why do we need to implement it?
  2. What are the rules for caching?
  3. How do you implement caching?

I’ll start by explaining what caching is, why we need to introduce caching in our application, and how to implement it.

1. What is caching and why do we need to implement it?

To understand what caching is and why we need it, consider the following scenario:

  • We are building an application in Python that will present the list of products to the end user.
  • More than 100 users will visit the application multiple times per day.
  • The application will be hosted on an application server and accessible over the Internet.
  • The information for the product will be stored in the database server.
  • So the application server queries the database for relevant records.

The following figure shows how our target application is configured:

The figure above illustrates how the application server retrieves data from the database server.

The problem

Retrieving data from a database is an I/O intensive operation. So it’s slow by nature. If the server needs to send requests frequently but the server can’t keep up, we can cache the response in the application’s memory.

Instead of querying the database every time, we can cache the results as follows:

The request for data must go through the network cable, and the response must go back through the network cable.

This is slow by nature. Therefore, we introduced caching.

We can cache the results to reduce computation time and save computer resources.

A cache is a temporary storage location. It works lazy-loaded.

At first, the cache is empty. When the application server fetches data from the database server, the data set populates the cache. From then on, subsequent requests can fetch data directly from the cache.

We also need to invalidate the cache in a timely manner to ensure that we are showing the latest information to the end user.

The next part of this article: cache rules.

2. Cache rules

In my opinion, there are three rules for caching.

Before enabling caching, we need to perform a key step — analyze the application.

Therefore, the first step before introducing caching in an application is to analyze the application.

Only then can we know how long each function takes to execute and how many times it is called.

In this article I explain the art of analysis, which I highly recommend.

Once the analysis is complete, we need to determine what we need to cache.

We need a mechanism to link the input to the output of the function and store it in memory. This is the first rule of caching.

2.1. Rule Number one:

The first rule is to ensure that the target function does take a long time to return the output, and that it is executed frequently but the output does not change much.

We don’t want to introduce caching for functions that don’t take a long time to complete, or that rarely get called in the application, or that return results change frequently in the source.

Please keep this important rule in mind.

Candidates for caching = functions that are called frequently, whose output does not change often and take a long time to execute.

For example, if a function is executed 100 times and it takes a long time to return a result, and it returns the same result for a given input, we can cache it.

Conversely, if the value returned by a function is updated every second in the source, but we only receive one request to execute the function every minute, we need to know if we need to cache the result. This is important because it can cause the application to send outdated data to the user. This example can help us understand whether caching is needed and whether different communication channels, data structures, or serialization mechanisms are needed to retrieve data faster, such as sending data over sockets using binary serializers or sending data over HTTP XML serialization.

In addition, it is important to know when to invalidate the cache and reload new data into the cache.

2.2. Rule Number two:

The second rule is to ensure that data can be retrieved from the cache faster than the target function can be executed.

We should only introduce caching if practicing caching has a positive impact on the time it takes to retrieve results.

Caching should be faster than fetching data from the current data source.

Therefore, it is critical to select an appropriate data structure (such as a dictionary or an LRU cache) as an instance of the cache.

2.3. Rule 3:

The third important rule is about memory footprint, which is often overlooked. Do you perform I/O operations (such as querying databases, network services) or CPU intensive operations (such as crunching numbers and performing in-memory calculations)?

When we cache data, the memory footprint of our application increases, so it is critical to choose the right data structure and cache only the data that needs to be cached.

Sometimes we have to query multiple tables to create an object of a class. However, we only need to cache the basic attributes in our application.

Caching has an impact on memory footprint.

For example, suppose we build a report panel to query the database and retrieve a list of orders. To give you a general picture, let’s assume that the panel displays only the order name.

Therefore, instead of caching the entire order object, we can cache only the name of each order. In general, the architect recommends creating a “skinny” data transfer object (DTO) with a __slots__ attribute to reduce memory footprint. Naming tuples or dataclass classes has the same effect. (The Dataclass class is a new feature in Python 3.7.)

The final section of this article Outlines the details of implementing caching.

3. How to implement cache?

There are several ways to implement caching.

We can build caches by creating local data structures in Python processes, or by using caches as servers that act as proxies and service requests.

There are built-in tools in Python, such as the Cached_property decorator in the FuncTools library. I’ll use it to introduce the implementation of caching. (Only available in Python 3.8 and later.)

The following code snippet illustrates how cached_property works:

from functools import cached_property


class FinTech:

    @cached_property
    def run(self) :
        return list(range(1.100))
Copy the code

So FinTech().run is now cached, and the list(range(1,100)) will only be generated once. In a real-world scenario, however, we rarely need to cache attributes.

Let’s look at something else.

3.1. Dictionary method

For simple use cases, we can create/use mapped data structures (such as dictionaries), keep the data in memory and make it globally accessible.

There are several ways to do this. The simplest way is to create a singleton module, such as config.py.

In config.py, we can create a dictionary type field that is populated once at the beginning.

Later we can use the dictionary fields to get the results.

For example, take a look at the code below.

Config.py has the cache attribute:

cache = {}
Copy the code

Imagine that our application would query Yahoo Finance’s web services using the get_prices(symbol, start_date, end_date) function to get the company’s historical prices.

The historical price does not change, so we do not need to query the web service every time we need the historical price. We can cache prices in memory.

In internal practice, the function get_prices(symbol, start_date, end_date) can check whether the data is in the cache before attempting to return the result.

Let me explain this policy in code.

The following get_prices function takes a parameter named companies.

  1. First, the function creates a variable for the start and end dates, with the start date set to yesterday and the end date set to 12 days ago.
  2. It then creates a file namedtarget_keyA tuple type variable of. It has a unique value and consists of modules, functions, and start and end dates.
  3. The function starts inconfig.cacheFind key in. If it finds it, it checkscacheWhether the target company name is included.
  4. If the cache contains the company name, it returns the price from the cache.
  5. iftarget_keyNot in the cache, it retrieves all companies through Yahoo Finance, saves the prices in the cache for future calls, and returns the prices.

Therefore, the basic concept is to check the target key in the cache, and if it doesn’t exist, fetch it from the source and store it in the cache before returning.

This is how the cache is built:

import datetime

import yfinance as yf

import config


def get_prices(companies) :
    end_date = datetime.datetime.now() - datetime.timedelta(days=1)
    start_date = end_date - datetime.timedelta(days=11)
    target_key = (__name__, '_get_prices_',
                  start_date.strftime("%Y-%m-%d"),
                  end_date.strftime("%Y-%m-%d"))

    if target_key in config.cache:
        cached_prices = config.cache[target_key]
        # cached_prices is a dictionary where the key is the company symbol and the value is the price.
        prices = {}
        for company in companies:
            # For each company, only the uncached prices are retrieved.
            if company in cached_prices:
                # We have cached the price.
                Set the price in a local variable.
                prices[company] = cached_prices[company]
            else:
                # Go to Yahoo Finance to get a price.
                yahoo_prices = yf.download(company, start=start_date, end=end_date)
                prices[company] = yahoo_prices['Close']
                cached_prices[company] = prices[company]
        return prices
    else:
        company_symbols = ' '.join(map(lambda x: x, companies))
        yahoo_prices = yf.download(company_symbols, start=start_date, end=end_date)
        Store it now in the cache for future use.
        prices_per_company = yahoo_prices['Close'].to_dict()
        config.cache[target_key] = prices_per_company
        return prices_per_company
Copy the code

The selected key above contains the start and end dates. You can also include the company name in the key, storing (company name, start date, end date, function name) as the key.

The key of the data structure must be unique, and it can be a tuple.

Historical prices don’t change, so it’s safe not to build logic that invalidates the cache in time.

The cache is a nested dictionary because its value is a dictionary. It speeds up lookups so that the time complexity of the operation is O(1).

Here’s a great article that explains the temporal complexity of Python data structures in an easy-to-understand way.

Sometimes keys get too long, it’s best to hash them using MD5, SHA, etc. However, using hashes can cause conflicts such that two strings produce the same hash.

We can also use memoisation.

Mnemonization is typically used for recursive function calls, with intermediate results stored in memory and returned when needed.

So I’ll introduce LRU.

3.2. The LRU algorithm

We can use Python’s built-in LRU feature.

LRU stands for the Least Recently Used algorithm. LRU can rely on the parameters of the function to cache the return value.

LRU is particularly useful in CPU-intensive recursive operations.

It is essentially a decorator: @lru_cache(maxsize, typed), which we can use to decorate our functions.

  • maxsizeTells the decorator the maximum size of the cache (the default is 128). If we don’t want to limit the size, we just set it toNone.
  • When caches compare input/output,typedIndicates whether the same value for different data types should be cached separately. (iftypedSet toTrue, function parameters of different types are cached separately. For example,f(3) 和 F (3.0)Will be treated as different calls with different results.

This is useful when we expect the same output from the same input.

Saving all of your application’s data to memory can also be confusing.

This can be a problem in distributed applications with multiple processes. In this case, it is not appropriate to cache all results in memory for all processes.

A good use case is to host the cache as a service when the application is running on a group of machines.

3.3. Service caching

The third option is to host the cached data as an external service. It acts as a proxy server for all requests and responses, and all applications can retrieve data through it.

Consider that we are building an application the size of Wikipedia that can handle 1,000 requests simultaneously and in parallel.

We need a caching mechanism and want to distribute the cache between servers.

You can use Memcached to cache data.

Memcached is very popular on Linux and Windows because:

  • It can be used to implement stateful memory caching.
  • It can even be distributed across servers.
  • It is simple to use, fast, and used by multiple large organizations across the industry.
  • Automatic expiration of cached data is supported.

We need to install a Python library called PymemCache.

Memcached requires that the data be stored as a string or binary. Therefore, we must serialize the cache object. When we want to retrieve them, we must deserialize them.

The following code snippet shows how to start and use Memcached: before executing the code, you need to download Memcached and start it. For how to create Serialiser and Deserialiser, see here.)

from pymemcache.client.base import Client


client = Client(host, serialiser, deserialiser)
client.set('blog', {'name': 'caching'.'publication': 'fintechexplained'})
blog = client.get('blog')
Copy the code

3.4. Problem: Invalidate cache

Finally, I want to quickly outline a scenario where the output of a function with the same input changes frequently, and we want to shorten the cache time of the results.

Consider two applications running on two different application servers.

The first application fetches data from the database server, and the second application updates the data in the database server. Data is fetched frequently, and we want to cache the data in the first application server:

Here are a few ways to solve it:

  • Each time a new record is stored, the application in the second application server can notify the first application server so that the cache can be flushed. It publishes messages to the first queue that an application can subscribe to.
  • The first application server can also make a lightweight call to the database server to find the last time the data was updated, which it can then use to determine whether the cache needs to be refreshed or fetched from.
  • We can also add a timeout to clear the cache so that it can be reloaded by the next request. It’s easy to implement, but not as reliable as my last option. It can go throughsignalLibrary implementation, we can subscribe to a handler tosignal.alarm(timeout)And, intimeoutWhen called, the cache in handler is cleared. We can also run background threads to clear the cache, but make sure you use the appropriate synchronization objects.

conclusion

Caching is an important concept that every Python programmer and data scientist needs to understand.

If you find any mistakes in your translation or other areas that need to be improved, you are welcome to the Nuggets Translation Program to revise and PR your translation, and you can also get the corresponding reward points. The permanent link to this article at the beginning of this article is the MarkDown link to this article on GitHub.

The Nuggets Translation Project is a community that translates quality Internet technical articles from English sharing articles on nuggets. The content covers Android, iOS, front-end, back-end, blockchain, products, design, artificial intelligence and other fields. If you want to see more high-quality translation, please continue to pay attention to the Translation plan of Digging Gold, the official Weibo, Zhihu column.