Signal is an important method of decoupling in flask/ Django. Flask’s signal relies on Blinker’s implementation, and Django’s signal is similar. The Blinker library is a powerful signal library implemented purely in Python with simple code. In this article we’ll start with Blinker and take a look at the signal mechanism for Python-Web development:

  • The blinker API
  • Blinker – the realization of the signal
  • The realization of the flask – signal
  • Django – the realization of the signal
  • Introduce weakref
  • summary
  • tip

Blinker profile

Blinker source code uses version 1.4, and the project structure is as follows:

file describe
base.py The core logic
_saferef.py Secure reference logic
_utilities.py Utility class

The blinker API

Examples of blinker API usage:

from blinker import signal

def subscriber1(sender):
    print("1 Got a signal sent by %r" % sender)

def subscriber2(sender):
    print("2 Got a signal sent by %r" % sender)

ready = signal('ready')
print(ready)
ready.connect(subscriber1)
ready.connect(subscriber2)
ready.send("go")
Copy the code

Example log output:

<blinker.base.NamedSignal object at 0x7f93a805ad00; 'ready'>
1 Got a signal sent by 'go'
2 Got a signal sent by 'go'
Copy the code

You can see that Signal is in publish/subscribe mode. Or, to put it more commonly, the center of the event:

  • ready = signal('ready')Create an event center named Ready
  • ready.connect(subscriber1)Add an event listener to the Ready event center
  • ready.send("go")Dispatch events to the Ready event center so that event listeners receive them and process them

The realization of the signal

Signal’s default singleton provides an out-of-the-box API:

class NamedSignal(Signal):
    """A named generic notification emitter."""

    def __init__(self, name, doc=None):
        Signal.__init__(self, doc)
        self.name = name

class Namespace(dict):

    def signal(self, name, doc=None):
        try:
            return self[name]
        except KeyError:
            return self.setdefault(name, NamedSignal(name, doc))

signal = Namespace().signal
Copy the code

It should be noted that the singleton of signal is bound to name. The same NamedSignal object is obtained by the same name. Different NamedSignal objects are obtained by different names.

Signal constructor: NamedSignal: receivers, receivers, receivers, receivers, receivers, receivers, receivers, receivers, receivers, receivers, receivers, receivers 2) Receiver ID- sender ID dictionary: receiver ID as key and sender ID set as value; Key = sender ID; value = receiver set;

ANY = symbol('ANY')

class Signal(object):

    ANY = ANY
    
    def __init__(self, doc=None)
        self.receivers = {}
        self._by_receiver = defaultdict(set)
        self._by_sender = defaultdict(set)
        ...
Copy the code

Signal’s connect function adds the message receiver, and you can see that the sender and receiver have a many-to-many relationship.

def connect(self, receiver, sender=ANY, weak=True): receiver_id = hashable_identity(receiver) receiver_ref = receiver sender_id = ANY_ID self.receivers.setdefault(receiver_id, receiver_ref) self._by_sender[sender_id].add(receiver_id) self._by_receiver[receiver_id].add(sender_id) del receiver_ref  return receiverCopy the code

Signal’s send function sends messages to all receivers that are interested in the sender:

def send(self, *sender, **kwargs): Sender = sender[0] # loop all receiver return [(receiver, receiver(sender, **kwargs)) for receiver in self.receivers_for(sender)] def receivers_for(self, sender): Receiver_id if sender_id in self._by_sender: # 2 set collection ids = (self) _by_sender [ANY_ID] | self. _by_sender [sender_id]) else: ids = self._by_sender[ANY_ID].copy() for receiver_id in ids: Receiver = self.receivers. Get (receiver_id) if receiver is None: continue # iterator yield receiverCopy the code

Signal uses the disconnect function to disconnect the receiver of the message:

def disconnect(self, receiver, sender=ANY):
    sender_id = ANY_ID
    receiver_id = hashable_identity(receiver)
    self._disconnect(receiver_id, sender_id)

def _disconnect(self, receiver_id, sender_id):
    if sender_id == ANY_ID:
        if self._by_receiver.pop(receiver_id, False):
            for bucket in self._by_sender.values():
                bucket.discard(receiver_id)
        self.receivers.pop(receiver_id, None)
    else:
        self._by_sender[sender_id].discard(receiver_id)
        self._by_receiver[receiver_id].discard(sender_id)
Copy the code

For the sake of understanding the signal mechanism, we will ignore the weakRef-related code and cover it later.

The realization of the flask – signal

Flask-signal relies on Blinker’s implementation:

# flask.signals.py

from blinker import Namespace

_signals = Namespace()

template_rendered = _signals.signal("template-rendered")
before_render_template = _signals.signal("before-render-template")
request_started = _signals.signal("request-started")
request_finished = _signals.signal("request-finished")
request_tearing_down = _signals.signal("request-tearing-down")
got_request_exception = _signals.signal("got-request-exception")
appcontext_tearing_down = _signals.signal("appcontext-tearing-down")
appcontext_pushed = _signals.signal("appcontext-pushed")
appcontext_popped = _signals.signal("appcontext-popped")
message_flashed = _signals.signal("message-flashed")
Copy the code

Flask prefabricated multiple signals using Blinker as you can see from the code above. Flask sends events to request_started when processing a request:

# flask.app.py

from .signals import request_started

def full_dispatch_request(self):
    ...
    request_started.send(self)
    ...
Copy the code

We can register event listeners in our own code like this:

def log_request(sender, **extra):
    sender.logger.debug('Request context is set up')

from flask import request_started
request_started.connect(log_request, app)
Copy the code

This makes it easy to use Signal to retrieve Flask data at various stages.

Django – the realization of the signal

Django-signal is a standalone implementation, but its pattern is very similar to Blinker’s. The Signal constructor creates an object that acts as the event center.

# django/dispatch/dispatcher.py

def _make_id(target):
    if hasattr(target, '__func__'):
        return (id(target.__self__), id(target.__func__))
    return id(target)

NONE_ID = _make_id(None)

# A marker for caching
NO_RECEIVERS = object()

class Signal:

    def __init__(self, providing_args=None, use_caching=False):
        """
        Create a new signal.
        """
        self.receivers = []
        self.lock = threading.Lock()
        self.use_caching = use_caching
        self.sender_receivers_cache = weakref.WeakKeyDictionary() if use_caching else {}
        self._dead_receivers = False
Copy the code

The core function of Connect is to build a unique identifier (receiver_id,sender_id) for the event listener and add it to the Array.

def connect(self, receiver, sender=None, weak=True, dispatch_uid=None):
    from django.conf import settings

    lookup_key = (_make_id(receiver), _make_id(sender))

    ref = weakref.ref
    receiver_object = receiver
    receiver = ref(receiver)

    with self.lock:
        if not any(r_key == lookup_key for r_key, _ in self.receivers):
            self.receivers.append((lookup_key, receiver))
Copy the code

The send function is similar to Blinker’s send:

def send(self, sender, **named):
    return [
        (receiver, receiver(signal=self, sender=sender, **named))
        for receiver in self._live_receivers(sender)
    ]

def _live_receivers(self, sender):
    with self.lock:
        senderkey = _make_id(sender)
        receivers = []
        for (receiverkey, r_senderkey), receiver in self.receivers:
            if r_senderkey == NONE_ID or r_senderkey == senderkey:
                receivers.append(receiver)
    ...
    non_weak_receivers = []
    for receiver in receivers:
        non_weak_receivers.append(receiver)
    return non_weak_receivers
Copy the code

Django-signal provides an additional receiver decorator to facilitate business use:

def receiver(signal, **kwargs):
    
    def _decorator(func):
        if isinstance(signal, (list, tuple)):
            for s in signal:
                s.connect(func, **kwargs)
        else:
            signal.connect(func, **kwargs)
        return func
    return _decorator
Copy the code

Djangos model packs the ModelSignal class and presets some signals:

class ModelSignal(Signal): def _lazy_method(self, method, apps, receiver, sender, **kwargs): from django.db.models.options import Options # This partial takes a single optional argument named "sender". partial_method = partial(method, receiver, **kwargs) if isinstance(sender, str): apps = apps or Options.default_apps apps.lazy_model_operation(partial_method, make_model_tuple(sender)) else: return partial_method(sender) def connect(self, receiver, sender=None, weak=True, dispatch_uid=None, apps=None): self._lazy_method( super().connect, apps, receiver, sender, weak=weak, dispatch_uid=dispatch_uid, ) ... Signal pre_init = ModelSignal(use_caching=True) post_init = ModelSignal(use_caching=True) pre_save = ModelSignal(use_caching=True) post_save = ModelSignal(use_caching=True) pre_delete = ModelSignal(use_caching=True) post_delete = ModelSignal(use_caching=True) m2m_changed = ModelSignal(use_caching=True) pre_migrate = Signal()Copy the code

The use of signal is explained in the comments for the Receiver decorator:

""" A decorator for connecting receivers to signals. Used by passing in the signal (or list of signals) and keyword arguments to connect:: @receiver(post_save, sender=MyModel) def signal_receiver(sender, **kwargs): ... @receiver([post_save, post_delete], sender=MyModel) def signals_receiver(sender, **kwargs): ... "" "Copy the code

This allows for some additional logical processing of MyModel using the signal mechanism, while avoiding hard coupling of the code.

Introduce weakref

After understanding the various implementations and uses of Signal, let’s go back to weakRef, another link in Blinker-Signal. Weakref can significantly improve Signal’s performance, as shown in the following example:

def test_weak_value_dict(cache): c_list = [] class C: def method(self): return ("method called!" , id(self)) c1 = C() c2 = C() c3 = C() c_list.append(c1) c_list.append(c2) c_list.append(c3) del c1, c2, c3 def do_cache(cache, name, target): cache[name] = target for idx, target in enumerate(c_list): do_cache(cache, idx, target) for k, v in cache.items(): print("before", k, v.method()) del c_list gc.collect() for x, y in cache.items(): print("after", x, y.method()) test_weak_value_dict({}) print("==" * 10) test_weak_value_dict(weakref.WeakValueDictionary())Copy the code

In test_weak_value_dict, three objects are created, placed in a list and cache, and then deleted and gc is performed. If the cache implementation is set, there will still be three objects in the cache after gc. If the cache is implemented using a WeakValueDictionary implementation, some objects are reclaimed. In an event center, memory continues to grow if the listener is canceled but cannot be freed.

before 0 ('method called! ', 140431874960640) before 1 ('method called! ', 140431874959440) before 2 ('method called! ', 140431874959968) after 0 ('method called! ', 140431874960640) after 1 ('method called! ', 140431874959440) after 2 ('method called! ', 140431874959968) ==================== before 0 ('method called! ', 140431875860416) before 1 ('method called! ', 140431875860128) before 2 ('method called! ', 140431876163136) after 2 ('method called! ', 140431876163136).Copy the code

Why does WeakValueDictionary keep the last data? Welcome to the comments section

Signal summary

The Blinker/Flask/Django signal is a pure Python message center, unlike the system signal we used in Gunicorn. Message centers, which can be used to decouple business logic, generally consist of three steps:

  • Register listeners
  • Distributed event
  • Logout listener

tip

Blinker provides an implementation reference for the singleton pattern, which I call a grouped singleton, where the same group name gives the same object instance:

class _symbol(object): def __init__(self, group): """Construct a new group symbol.""" # Self.__group__ = self. Group = group def __reduce__(self): return symbol, (self.group,) def __repr__(self): return self.group _symbol.__group__ = 'symbol' class symbol(object): >>> symbol('foo') is symbol('foo') True >>> symbol('foo') foo """ symbols =  {} def __new__(cls, group): try: return cls.symbols[group] except KeyError: Return cls.symbols.setdefault(group, _symbol(group)) ANY = symbol('ANY') # singletonCopy the code

Reference links:

  • pythonhosted.org/blinker/
  • Stackify.com/python-garb…
  • www.cnblogs.com/TM0831/p/10…
  • www.geeksforgeeks.org/weak-refere…
  • pymotw.com/2/weakref/