No.1 Everything is an object

Java is known for its emphasis on “everything is an object”, but object orientation in Python is more thorough than Java, because classes in Python are objects, functions are objects, and Python code and modules are objects.

  • Functions and classes in Python can be assigned to a variable
  • Python functions and classes can be stored in collection objects
  • In Python functions and classes can be passed as arguments to a function
  • Functions and classes in Python can be returned as values

Step.1

Step.2

Step.3

Step.4

No.2 Relationships between keywords Type, object, and class

In Python, an instance of object is type. Object is a top-level class, with no base class. An instance of Type is type, and the base class of Type is Object. The base class of the built-in types in Python is Object, but they are instantiated from Type, and the values are instantiated from the built-in types. In python2. x, user-defined classes that do not explicitly specify a base class default to no base class. In Python3.x, the base class is specified as object.

No.3 Python built-in types

In Python, objects have three characteristic properties:

  • Address in memory, usedid()Function to view
  • Type of object
  • Object default value

Step 1 type of None

When the Python interpreter starts, a None object of type None is created, and None has only one object globally.

Step.2 Indicates the value type

  • Ini type
  • A float
  • Complex type
  • Bool type

Step.3 Iteration type

In Python, iterative types can be traversed using loops.

Step.4 Sequence type

  • list
  • tuple
  • str
  • array
  • range
  • Bytes, bytearray, memoryVie (binary sequence)

Step.5 Mapping type

  • dict

Step.6 Set type

  • set
  • frozenset

Step.7 Context management type

  • With statement

Step.8 Other types

  • The module
  • class
  • The instance
  • function
  • methods
  • code
  • object
  • The type object
  • Ellipsis (ellipsis)
  • notimplemented

No.4 Magic function

Magic functions in Python start with a double underscore and end with a double underscore

No.5 Duck type and white goose type

Duck types are a style of inference in programming that focuses on how objects are used rather than on the types themselves. Duck types work like polymorphism but have no inheritance. The concept of the duck type comes from: “A bird can be called a duck when it is seen to walk like a duck, swim like a duck, and quack like a duck.”

White goose type refers to as long asclsIs the abstract base class, i.eclsClass is the yuanabc.ABCMeta, can be usedisinstance(obj, cls) 。

No.6 Inheritance relationships between protocols, abstract base classes, ABC modules, and sequences

  • Protocol: An informal interface in Python that allows Python to implement polymorphism. Protocols are informal, non-mandatory, and defined by conventions and documentation.
  • Interface: An abstraction that an entity provides itself to the outside world (possibly another entity) to separate external communication methods from internal operations so that they can be modified internally without affecting the way other entities interact with it.

We can use monkey patches to implement protocols, so what is monkey patches?

Monkey patching is to modify the module or class at runtime without modifying the source code to implement the target protocol interface operation. This is called monkey patching.

Monkey patches are derived from the Zope framework, where developers often fix Zope bugs by adding new patches at the end of the program. These miscellaneous patches are called Guerilla Patches, later gorllia, and later monkey.

The main functions of the monkey patch are:

  • Replace methods, attributes at run time
  • Add functionality to the program itself without modifying the source code
  • Add patches to run-time objects, not to the source code on disk

Application case: Suppose you write a large project, use JSON module everywhere to parse JSON file, but later found that UJSON is better than JSON performance, modify the source code is to modify many places, so only need to add in the program entry:

Python’s abstract base classes have one important practical advantage: you can use the Register class method to “declare” a class in end user code as a “virtual” subclass of an abstract base class. (To do this, the registered class must 腨 satisfy the method name and signature requirements of the abstract class, and most importantly, the underlying semantic contract; However, you don’t need to know about abstract base classes to develop that class, let alone inherit from them. Sometimes, registration is not even necessary for the abstract class to recognize subclasses. Resist the urge to create abstract base classes. Misuse of abstract base classes can be disastrous, showing that the language is too focused on superficial form.

  • Abstract base classes cannot be instantiated (objects cannot be created) and are usually inherited by subclasses that override virtual functions to implement concrete interfaces.
  • Determines the type of an object
  • Enforce that subclasses must implement certain methods

Definition and use of abstract base classes

Note that between Python 3.0 and Python3.3, the syntax for inheriting abstract base classes isclass ClassName(metaclass=adc.ABCMeta), other versions are:class ClassName(abc.ABC).

  • UML class diagrams for the various abstract base classes in the collections.abc module

No.7 Difference between isinstence and Type

Python is full of me

No.8 Class variables and instance variables

  • Instance variables can only be called from an instance of a class
  • Modify the properties of the object created by the template object. The properties of the template object do not change
  • Modifying the properties of a template object changes the properties of the object created by the template object

No.9 The order in which class and instance attributes and methods are found

  • Before Python 2.2, there were only classical classes, but Python2.7 will still be compatible with classical classes, Python3.x will only use modern classes, and Python versions before that will be compatible with modern classes
  • Python 2.2 and previous classes have no base classes, and Python’s new classes need to explicitly inherit from themobjectIs inherited by default even if it is not explicitly inheritedobject
  • Classical classes use left-to-right depth-first matching in class multiple inheritance. The new class uses the C3 algorithm
  • The classical class does not have MRO and instance.mro() calls

Assume the following inheritance:

When the say_hello() method of A is called, the system looks for the say_hello() method in B. If the say_hello() method is not found in B, the system looks for the say_hello() method in D. It is obvious that the say_hello() method exists in D.

When the say_hello method is called in an instance of A, the system looks for the say_hello method in B first. Since the say_hello method is not defined in class B, it looks for the say_hello method in D, and the say_hello method is not defined in class D either. The system considers the say_hello method undefined, but it is defined in C. So consider using BFS (breadth first search algorithm), then the problem goes back to the first inheritance relationship, assuming that C and D have the same name method, when calling the method of an instance of A, it should look in B first, it should call the method in D, but when using BFS, the method in class C will override the method in class D. In Python 2.3 and later, the C3 algorithm is used:

The second inheritance order after using the C3 algorithm:

The first inheritance order after using the C3 algorithm:

Only the function and evolution history of algorithm are introduced here

No.10 Class methods, instance methods, and static methods

Instance methods can only be called from an instance of a class; A static method is an independent, stateless function that is tied to the namespace of its class. Class methods are used to retrieve data maintained in the class, for example:

No.11 Data encapsulation and private properties

The double underscore + attribute name implementation in Python is similar to the private modifier in static languages for data encapsulation.

No.12 Python’s introspection mechanism

Introspection is an act of self-examination. In computer programming, introspection refers to the ability to examine something to determine what it is, what it knows and what it can do. Introspection provides programmers with great flexibility and control.

  • Dir ([obj]) : Returns an sorted list of attribute names of any object passed to it (some special attributes may not be included)
  • Getattr (obj, attr) : Returns any property of any object. Calling this method returns the value of the property in obj named attr value
  • . .

No. 13 super function

One difference between python3. x and python2. x is that Python3 can use super().xxx directly instead of super(type[, object-or-type]).xxx.

The super() function is used to call the method of the next class in MRO (class method parsing order table).

No. 14 Mixin inheritance

Mixin classes are designed to be mixed as functionality into classes that inherit from Mixin. When implementing multiple inheritance using Mixin classes, note:

  • Mixin classes must represent some kind of functionality
  • Single responsibility, if you want to have more than one function, you need to design multiple Mixin classes
  • Independent of subclass implementation, Mixin classes exist simply to add subclass functionality
  • This works even if subclasses do not inherit from the Mixin class

Python is full of me

No.15 Context manager with statement simplified with contextlib

Common exception catching mechanisms:

With simplifies exception catching

No.16 Classification of sequence types

  • Container sequence: list tuple deque
  • Flat sequence: STR bytes bytearray array.array
  • Mutable sequence: List deque bytearray Array
  • Immutable sequence: STR tuple bytes

No.17 +, +=, and extend() differ in application scenarios

Let’s start with the test case:

From the source code snippet:

No.18 maintains a sorted sequence using Bisect

No.19 Deque

A deque is a dual-ended queue in Python that inserts data at both ends with O(1)O(1)O(1) efficiency, located in the Collections module.



Deque class source:

deque

When a list stores data, the internal implementation is an array, which is fast to find, but slow to insert and delete data. The internal implementation of a deque double-ended list is a double-ended queue. Deuque applies to queues and stacks and is thread-safe.

Deque provides append() and pop() functions to add and pop data at the end of the deque, and appendleft() and Popleft () functions to add and pop elements at the head of the deque. The time complexity of these four functions is O(1)O(1)O(1), but the time complexity of list is as high as O(n)O(n)O(n).

Create a deque queue

The source code

ChainMap

Used to merge multiple dictionaries.

The application case

The source code

UserDict

UserDict is a subclass of MutableMapping and Mapping. It inherits two important methods, mutableMapping. update and mapping. get.

The application case

The source code

No.20 Variables and garbage collection mechanisms in Python

Python and Java variables are fundamentally different. Python variables are Pointers. When the Python interpreter executes “number=1”, it creates an int in memory and points “number” to the memory address of the int. For example:

The difference between == and IS is that the former checks whether the value is equal, while the latter checks whether the object ID is equal.

Python has an optimization mechanism calledinternSmall integers, small strings, which are often used, are created at run time and are globally unique.

The Python del statement is not the same as the C++ delete statement. The Python del statement removes the reference to the object. The Python virtual machine will delete the object when there is no reference to the object.

No.21 Python metaclass programming

Property Dynamic attribute

In Python, add for functions@propertyDecorators allow functions to be accessed as variables.

Use of the __getattr__ and __getAttribute__ functions

__getattr__ calls this function when the attribute is not found.

__getAttribute__ calls this method before calling the attribute.

Attribute descriptor

Implement __get__(), __set__(), and __delete__() as property descriptors in a class.

Data property descriptor

Non-data attribute descriptor

In Python’s new classes, all access to object attributes calls the __getAttribute__ () method, which allows us to customize the access behavior when accessing objects, with the caution of infinite recursion. __getattriubte__() is the entry point for all method and attribute look-ups. When called, the corresponding attribute value or object is searched in __dict__ according to certain rules. If none is found, the __getattr__() method is called. The corresponding __setattr__() and __delattr__() methods use the assignment behavior from defining an attribute and the behavior used to handle deleting an attribute, respectively. The concept of descriptors was introduced in Python 2.2. __get__(), __set__(), and __delete__() define the actions of fetching, setting, and deleting the value of a descriptor, respectively.

  • It is worth noting that as long as any of these three methods are implemented, they are descriptors.
  • Only to realize__get__ ()Method descriptors are called non-data descriptors and can only be read after initialization.
  • At the same time to realize__get__()and__set__()Methods are called data descriptors, and properties are read and write.

Priority rules for property access

Attributes of objects are typically stored in __dict__, and __getAttribute__ () implements the rules for attribute access in Python.

Given that there is an instance of obj, the search for the attribute number in obj looks like this:

  • Search the list of base classestype(b).__mro__Until the property is found and assigned todescr.
  • judgedescrIs called if it is a data descriptordescr.__get__(b, type(b))And returns the result.
  • If it is something else (non-data descriptor, common attribute, type not found), the instance is looked upobjInstance property of, which isobj.__dict__.
  • If theobj.__dict__If the attribute is not found, it is returned todescrOn the judgment of.
  • If you judge againdescrIf the type is a non-data descriptor, it is calleddescr.__get__(b, type(b))And returns the result to end the execution.
  • ifdescrIs a common property that returns the result directly.
  • If it is not found the second time, it is nullAttributeErrorException, and end the search.

Represented by a flow chart:

__new__()and__init__()The difference between

  • __new__()The function controls the generation of an object and is called before it is generated on the object.
  • __init__()The function is used to refine an object and is called after the object is generated.
  • if__new__()If a function does not return an object, it is not called__init__()Function.

Custom metaclasses

In Python everything is an object. A class describes how to generate an object. In Python a class is an object because it has the ability to create objects. This so-called class object is created when the Python interpreter executes a class statement. Since a class is an object, it can be created dynamically. Here we use the type() function. Here is the source of the constructor for this function:

It follows that,type()Receiving a description of a class returns a class.

Metaclasses are used to create classes because they are also objects.type()Classes can be created becausetyep()It’s a metaclass from which all classes in Python are created. In Python, we can pass an object’s__class__Property to determine which class the object was created by. When Python creates an object of a class, Python will look it up in that class__metaclass__Properties. If you find it, use it to create an object, if you don’t find it, look it up in the parent class, if you still don’t find it, look it up in the module, and if you haven’t found it along the way, use ittype()To create. To create a metaclass, write:

Use metaclasses to create apis

The main use of metaclases is to create apis, such as the ORM framework in Python.

Tim Peters, Python Leader:

“Metaclasses are magic of depth, and 99% of users shouldn’t have to worry about it at all. If you want to figure out whether you need a metaclass at all, you don’t need it. The people who actually use metaclasses know exactly what they need to do and don’t have to explain why they use metaclasses at all.”

No.22 Iterators and generators

When there are too many elements in a container to read all of them into memory, you need an algorithm to calculate the next element so you don’t have to create a very large container, and generators do just that.

Generators in Python use yield to return values, and yield pauses on each call, so the generator does not execute all at once. It evaluates when the result is needed. When the function reaches yield, it returns the value and saves the current execution state, that is, the function is suspended. We can use the next() and send() functions to restore the generator, replacing the list derivation [] with () to become a generator:

It’s worth noting that we don’t normally use the next() method to get elements, but instead use the for loop. When using the while loop, you need to catch the generation of StopIteration exceptions.

Each stack frame has its own data stack and block stack. Since these stack frames are stored in heap memory, the interpreter has the ability to interrupt and restore stack frames:

This is the basis for generators. Whenever we get the generator object anywhere, we can start or pause the generator, because the stack frame exists independently of the caller, which is the theoretical basis of the coroutine.

Iterators are a different way to access elements in a collection than the for loop. They are commonly used to iterate over data. Iterators provide a lazy way to access data.

There are several types of for loops that can be used:

  • Set data type
  • Generators, including generators and withyieldGenerator function of

These objects that can be called directly by the for loop are called iterables, and you can use isinstance() to determine whether an object is Iterable. Set data types such as list, dict, and STR are Iterable but not iterators, and an Iterator can be obtained from iter(). The difference between send() and next() is that send() can pass parameters to the yield() expression. In this case, the parameters passed are the yield expression values, whereas the yield parameters are the values returned to the caller. That is, SEND can force changes to the last yield expression value.

Python is full of me