Clean Code in Python Chapter 5 Using Decorators to Improve Our Code
- Learn how decorators work in Python
- Learn how to implement decorators that apply to functions and classes
- Implement decorators efficiently, avoiding common execution errors
- Analyzing how to avoid code duplication with decorators (DRY)
- How can research decorators contribute to separation of concerns
- Example analysis of excellent decorators
- Review common situations, idioms, or patterns to know when decorators are the right choice
Although you typically see decorators decorating methods and functions, you can actually decorate any type of object, so we’ll explore decorators that apply to functions, methods, generators, and classes.
Also be careful not to confuse decorators with Decorator patterns.
Function to decorate
Functions are probably the simplest representation of a Python object that can be decorated. We can use decorators on functions to achieve all sorts of logic — we can validate parameters, check preconditions, completely change behavior, modify signatures, cache results (create a stored version of the original function), and so on.
As an example, we will create a basic decorator that implements a retry mechanism, controls specific domain-level exceptions and retries a certain number of times:
# decorator_function_1.py
import logging
from functools import wraps
logger = logging.getLogger(__name__)
class ControlledException(Exception):
"""A generic exception on the program's domain."""
pass
def retry(operation):
@wraps(operation)
def wrapped(*args, **kwargs):
last_raised = None
RETRIES_LIMIT = 3
for _ in range(RETRIES_LIMIT):
try:
return operation(*args, **kwargs)
except ControlledException as e:
logger.info("retrying %s", operation.__qualname__)
last_raised = e
raise last_raised
return wrapped
Copy the codeYou can ignore @wraps for a moment and take a look at the Retry decorator use example:
@retry
def run_operation(task):
"""Run a particular task, simulating some failures on its execution."""
return task.run()
Copy the codeThis can be done because the decorator is just a syntactic sugar provided that is essentially equivalent to run_operation = retry(run_operation) the more common timeout retry.
Defines a decorator with parameters
Let’s use an example to elaborate on the process of accepting parameters. Suppose you want to write a decorator that adds logging to a function and allows the user to specify logging levels and other options. Here is an example of the decorator definition and use:
from functools import wraps
import logging
def logged(level, name=None, message=None):
""" Add logging to a function. level is the logging level, name is the logger name, and message is the log message. If name and message aren't specified, they default to the function's module and name. """
def decorate(func):
logname = name if name else func.__module__
log = logging.getLogger(logname)
logmsg = message if message else func.__name__
@wraps(func)
def wrapper(*args, **kwargs):
log.log(level, logmsg)
return func(*args, **kwargs)
return wrapper
return decorate
# Example use
@logged(logging.DEBUG)
def add(x, y):
return x + y
@logged(logging.CRITICAL, 'example')
def spam():
print('Spam! ')
Copy the codeAt first glance, this implementation looks complicated, but the core idea is simple. The outermost function logged() accepts arguments and acts them on the inner decorator function. The inner function invocation () takes a function as an argument and places a wrapper on top of it. The key point here is that wrappers can use arguments passed to logged().
Defining a wrapper that accepts arguments can seem complicated mainly because of the underlying call sequence. In particular, if you have this code:
@decorator(x, y, z)
def func(a, b):
pass
Copy the codeThe decorator processing is equivalent to the following call;
Def func(a, b): pass func = decorator(x, y, z)(func) The return result of a decorator(x, y, z) must be a callable object that takes a function as an argument and wraps itCopy the codeClass decoration
Some people argue that decorator classes are more complex things, and that such a scheme might compromise readability. Because we declare some properties and methods in the class, the decorator may change their behavior to render a completely different class.
This assessment is correct in cases where the technology is heavily abused. Objectively, this is no different from decorating functions; After all, classes are just another type of object in the Python ecosystem, just like functions. We’ll review the pros and cons of this issue together in the section titled “Decorators and Separation of concerns,” but for now, we’ll explore the benefits of class decorators:
- Code reuse and DRY. A case in point is a class decorator that forces multiple classes to conform to a particular interface or standard (which can be applied to multiple classes by checking only once in the decorator)
- You can create smaller or simpler classes and enhance them with decorators
- The transformation logic of a class will be easier to maintain, rather than using more complex (and often naturally discouraged) methods such as metaclasses
Reviewing the event system for the monitoring platform, we now need to transform the data for each event and send it to an external system. However, each type of event may have its own particularities when it comes to choosing how to send data.
In particular, the event of a login may contain sensitive information, such as login information that needs to be hidden, and other fields such as time stamps that may need to be displayed in a specific format.
class LoginEventSerializer:
def __init__(self, event):
self.event = event
def serialize(self) -> dict:
return {
"username": self.event.username,
"password": "**redacted**"."ip": self.event.ip,
"timestamp": self.event.timestamp.strftime("%Y-%m-%d% H: % M"),}
class LoginEvent:
SERIALIZER = LoginEventSerializer
def __init__(self, username, password, ip, timestamp):
self.username = username
self.password = password
self.ip = ip
self.timestamp = timestamp
def serialize(self) -> dict:
return self.SERIALIZER(self).serialize()
Copy the codeHere, we declare a class that maps directly to the login event, including its logic — hide the password field, and format the timestamp as needed.
While this approach works and seems like a good option, over time, when we want to extend our system, we find some problems:
- Too many classes: As the number of events increases, the number of serialized classes increases by the same order of magnitude because they are mapped one by one.
- The solution is not flexible enough: if we need to reuse part of a component (for example, if we need to hide a password in another event), we have to extract it into a function and call it repeatedly from multiple classes, which means we are not doing code reuse.
- The Boilerplate: serialize() method must appear in all event classes, calling the same code. Although we can extract it into another class (creating mixins), Although we can extract this into another class (creating a mixin), It does not seem like a good use of inheritance.
Another solution is to be able to construct objects dynamically, given a set of filters (conversion functions) and an event instance, that can serialize their fields through filters. Then, we just need to define the functions that convert each type of field and create the serializer by combining many of these functions.
Once we have this object, we can decorate the class to add the serialize() method, which will call only the Serialization objects themselves:
def hide_field(field) -> str:
return "**redacted**"
def format_time(field_timestamp: datetime) -> str:
return field_timestamp.strftime("%Y-%m-%d %H:%M")
def show_original(event_field):
return event_field
class EventSerializer:
def __init__(self, serialization_fields: dict) -> None:
self.serialization_fields = serialization_fields
def serialize(self, event) -> dict:
return {
field: transformation(getattr(event, field))
for field, transformation in self.serialization_fields.items()
}
class Serialization:
def __init__(self, **transformations):
self.serializer = EventSerializer(transformations)
def __call__(self, event_class):
def serialize_method(event_instance):
return self.serializer.serialize(event_instance)
event_class.serialize = serialize_method
return event_class
@Serialization(
username=show_original,
password=hide_field,
ip=show_original,
timestamp=format_time,
)
class LoginEvent:
def __init__(self, username, password, ip, timestamp):
self.username = username
self.password = password
self.ip = ip
self.timestamp = timestamp
Copy the codeTo be continued…