Table of Contents generated with DocToc

  • Effective Python
    • Pythonic
      • The list of cutting
      • List derivation
      • The iteration
      • A reverse iterative
      • try/except/else/finally
    • function
      • Use decorators
      • Using generators
      • iterable
      • Using positional parameters
      • Use keyword arguments
      • Default values for parameters
    • class
      • __slots__
      • __call__
      • @classmethod & @staticmethod
      • Create a context manager

Effective Python

Some of them are from books: Effective Python & Python3 Cookbook, but have also been modified and added with my own understanding and best practices in use

The list of cutting

list[start:end:step]

  • If you are cutting from the beginning of the list, ignore the 0 of the start bit, for examplelist[:4]
  • If you cut all the way to the end of the list, the 0 of the end bit is ignored, for examplelist[3:]
  • When cutting a list, there is no problem even if the start or end index is crossed
  • List slicing does not change the original list. When indexes are left blank, a copy of the original list is generated
b = a[:]
assert b == a and b is not a # trueCopy the code

The iteration

  • Used when you need to get indexenumerate
  • enumerateYou can accept the second argument as added during iterationindexIn the numerical
list = ['a', 'b', 'c', 'd']

for index, value in enumerate(list):
    print(index)
# 0
# 1
# 2
# 3

for index, value in enumerate(list, 2):
    print(index)
# 2
# 3
# 4
# 5Copy the code

  • withzipIterate over both iterators at the same time
List_a = [' a ', 'b', 'c', 'd'] list_b = [1, 2, 3] # though is not the same as the length of the list, but as long as there is a list of running out, For letter, number in zip(list_a, list_b): print(letter, number) # a 1 # b 2 # c 3Copy the code

  • zipTraversal returns a tuple
a = [1, 2, 3]
b = ['w', 'x', 'y', 'z']
for i in zip(a,b):
    print(i)

# (1, 'w')
# (2, 'x')
# (3, 'y')Copy the code

  • aboutforandwhileAfter the cycleelseblock
    • cycleThe end of the normalAnd then it callselseThe code in
    • Loop throughbreakOut of the loop, it will not be executedelse
    • Execute immediately if the sequence to be traversed is emptyelse
for i in range(2):
    print(i)
else:
    print('loop finish')
# 0
# 1
# loop finish

for i in range(2):
    print(i)
    if i % 2 == 0:
        break
else:
    print('loop finish')
# 0Copy the code

try/except/else/finally

  • iftryIf no exception occurs in theelseThe code in
  • elseWill be infinallyRun before
  • It will be executed eventuallyfinallyAnd can be cleaned up in it

Use decorators

Decorators are used to modify an existing function without changing the original function code. A common scenario is to add a debug sentence, or to add log monitoring to an existing function

Here’s an example:

def decorator_fun(fun):
    def new_fun(*args, **kwargs):
        print('current fun:', fun.__name__)
        print('position arguments:', args)
        print('key arguments:', **kwargs)
        result = fun(*args, **kwargs)
        print(result)
        return result
    return new_fun

@decorator_fun
def add(a, b):
    return a + b

add(3, 2)
# current fun: add
# position arguments: (3, 2)
# key arguments: {}
# 5Copy the code

In addition, we can write a decorator that receives arguments, in effect nesting a function on top of the original decorator:

def read_file(filename='results.txt'): def decorator_fun(fun): def new_fun(*args, **kwargs): result = fun(*args, **kwargs) with open(filename, 'a') as f: F. Write (result + '\n') return result return new_fun return decorator_fun # def @read_file(filename='log.txt') def  add(a, b): return a + bCopy the code

But there’s a problem with using decorators like the one above:

@decorator_fun
def add(a, b):
    return a + b

print(add.__name__)
# new_funCopy the code

This means that the original function has been replaced by the new_fun function in the decorator. Calling a decorated function is equivalent to calling a new function. When you view the arguments, comments, and even the function name of the original function, you only see the information about the decorator. To solve this problem, use the functools.wraps method that comes with Python.

stackoverflow: What does functools.wraps do?

Functools. wraps is a hack method that acts as a decorator for the function that will be returned inside the decorator. In other words, it is a decorator of a decorator, and it takes the original function as an argument. Its purpose is to retain various information about the original function, so that when we view the decorated original function, it can be exactly the same as the original function.

from functools import wraps

def decorator_fun(fun):
    @wraps(fun)
    def new_fun(*args, **kwargs):
        result = fun(*args, **kwargs)
        print(result)
        return result
    return new_fun

@decorator_fun
def add(a, b):
    return a + b

print(add.__name__)
# addCopy the code

In addition, sometimes our decorators may do more than one thing, and events should be separated out as additional functions. But because it might be related only to the decorator, you can construct a decorator class at this point. The principle is very simple, mainly is to write the __call__ method in the class, so that the class can be called like a function.

from functools import wraps

class logResult(object):
    def __init__(self, filename='results.txt'):
        self.filename = filename

    def __call__(self, fun):
        @wraps(fun)
        def new_fun(*args, **kwargs):
            result = fun(*args, **kwargs)
            with open(filename, 'a') as f:
                f.write(result + '\n')
            return result
        self.send_notification()
        return new_fun

    def send_notification(self):
        pass

@logResult('log.txt')
def add(a, b):
    return a + bCopy the code

Using generators

Consider using generators to override functions that return lists directly

Def get_a_indexs(string) def get_a_indexs(string) def get_a_indexs(string): result = [] for index, letter in enumerate(string): if letter == 'a': result.append(index) return resultCopy the code

There are a few minor problems with this approach:

  • Called each time a qualified result is obtainedappendMethods. But we’re not really focused on the method at all, it’s just a means to an end, it’s really just a matter ofindexIt’s good
  • The returnedresultI can continue to optimize
  • The data is thereresultInside, if the amount of data is very large, it will take up more memory

Therefore, it is better to use a generator. A generator is a function that uses yield expressions. When a generator is called, it does not actually execute. Instead, it returns an iterator that advances the generator to the next yield expression each time the built-in next function is called on the iterator:

def get_a_indexs(string):
    for index, letter in enumerate(string):
        if letter == 'a':
            yield indexCopy the code

Once you get a generator, you can iterate over it normally:

String = 'this is a test to find a\' index' indexs = get_a_indexs(string) # Print (I) # try: while True: print(next(indexs)) except StopIteration: print(' Finish! ') # The generator raises a StopIteration error if it continues through next() after fetching the value, but this error is automatically caught by iterating through the for loopCopy the code

If you still need a list, you can call the list method with the result of the function call as an argument

results = get_a_indexs('this is a test to check a')
results_list = list(results)
Copy the code

Using positional parameters

In some cases, the number of arguments a method takes may not be certain, such as a summation method that takes at least two arguments:

def sum(a, b): Return a + b # sum(1, 2) # 3 Sum (1, 2, 3, 4, 5) # sum() takes 2 positional arguments but 5 were givenCopy the code

For functions that receive a variable number of arguments and do not care about the order in which they are passed in, the positional argument *args should be used:

def sum(*args): result = 0 for num in args: Num return result sum(1, 2) # 3 sum(1, 2, 3, 4, 5) # 15 Sum (*[1, 2, 3, 4, 5]) # 15Copy the code

Note, however, that the variable length argument args needs to be converted to a tuple before being passed to the function. This means that if you pass a generator into a function as an argument, the generator will first iterate over it, converting it to a tuple. This can consume a lot of memory:

Def get_nums(): for num in range(10): yield num nums = get_nums() sum(*nums) # 45 #Copy the code

Default values for parameters

It’s a cliche: the default value of a function is set only once when the module is loaded and the function definition is read

That is, if you assign a dynamic value to a parameter (such as [] or {}), then if you assign another parameter to the parameter when calling the function later, the default value defined previously will change to the value assigned last time:

def get_default(value=[]):
    return value

result = get_default()
result.append(1)
result2 = get_default()
result2.append(2)
print(result) # [1, 2]
print(result2) # [1, 2]Copy the code

Therefore, it is more recommended to use None as the default argument and assign it after checking within the function:

def get_default(value=None): if value is None: return [] return value result = get_default() result.append(1) result2 = get_default() result2.append(2) print(result) #  [1] print(result2) # [2]Copy the code

__slots__

By default, Python uses a dictionary to hold instance attributes of an object. This allows us to dynamically add new attributes to class instances at runtime:

test = Test()
test.new_key = 'new_value'Copy the code

However, this dictionary wastes extra space – many times we don’t create as many attributes. So __slots__ tells Python not to use dictionaries but fixed collections to allocate space.

__slots__ = ['name', 'value'] def __init__(self, name=' Test ', value='0'): Test. new_key = 'new_value' # AttributeError: self.name = name self.value = value test = test () 'Test' object has no attribute 'new_key'Copy the code

@classmethod & @staticmethod

Information:

@classMethod is similar to @staticMethod, but they are used in different scenarios.

  • All ordinary methods inside a class are defined asselfAs the first argument, the instance scope is passed into the method when called through the instance.
  • @classmethodIn order toclsAs the first argument, represents passing in the scope of the class itself. Whether called from a class or an instance of a class, the first argument passed in by default will be the class itself
  • @staticmethodYou don’t need to pass in default arguments, just like a normal function

To see their usage scenarios through examples:

Suppose we need to create a class named Date to store year/month/day data

class Date(object):
    def __init__(self, year=0, month=0, day=0):
        self.year = year
        self.month = month
        self.day = day

    @property
    def time(self):
        return "{year}-{month}-{day}".format(
            year=self.year,
            month=self.month,
            day=self.day
        )Copy the code

This code creates the Date class, which sets the day/month/year property at initialization, and sets a getter for property, which gets the stored time through time after instantiation:

date = Date('2016', '11', '09')
date.time # 2016-11-09Copy the code

But what if we want to change the way attributes are passed in? After all, it’s annoying to pass in year/month/day at initialization. Can you find a way to create a Date instance by passing in a string like 2016-11-09 without changing existing interfaces and methods?

You might think of something like this:

date_string = '2016-11-09'
year, month, day = map(str, date_string.split('-'))
date = Date(year, month, day)
Copy the code

But not good enough:

  • An extra method is written outside the class, and each time you format it, you get the parameters
  • This method also only followsDateClass about
  • Did not solve the problem of passing too many arguments

We can create a new format string inside the class using @classMethod and return the method of the instance of the class:

Def from_string(CLS, string) def from_string(CLS, string) Year, month, day = map(STR, string.split('-')) # Date = CLS (year, month, day) return dateCopy the code

This way, we can create an instance by calling the from_string method from the Date class without invading or modifying the old instantiation:

Date_old = date ('2016', '11', '09')Copy the code

Benefits:

  • in@classmethodInside, can passclsArgument, obtaining the same convenience as when calling classes externally
  • The method can be further encapsulated here to improve reusability
  • More in line with object-oriented programming

@staticMethod, because it is similar to a normal function, can put the helper method associated with this class in the class as @StaticMethod and call the method directly from the class.

# staticmethod @staticmethod def is_month_validate(month) return int(month) <= 12="" and="" int(month)="">= 1Copy the code

After placing the date-related helper class functions inside the Date class as @StaticMethod methods, these methods can be called from the class:

month = '08'
if not Date.is_month_validate(month):
    print('{} is a validate month number'.format(month))Copy the code

Create a context manager

Context manager, colloquial description is: before the code block is executed, the preparation work; After the execution of the code block is complete, do the finishing touches. The with statement often comes with a context manager. Classic scenarios include:

with open('test.txt', 'r') as file:
    for line in file.readlines():
        print(line)Copy the code

With the with statement, the code opens the file and automatically closes the file at the end of the call or when an exception occurs in the read, completing the processing of the file after it has been read or written. Without the context manager, it would look like this:

file = open('test.txt', 'r')
try:
    for line in file.readlines():
        print(line)
finally:
    file.close()Copy the code

Is it tedious? So the advantage of using a context manager is that it automatically handles the work when a block of code starts and finishes by calling our preset callbacks. By customizing the class’s __enter__ and __exit__ methods, we can customize a context manager.

class ReadFile(object): def __init__(self, filename): self.file = open(filename, 'r') def __enter__(self): return self.file def __exit__(self, type, value, traceback): Traceback = self.file.close() # return True = do not throw an error # Otherwise the error will be returned True by the with statementCopy the code

The call can then be made as follows:

with ReadFile('test.txt') as file_read:
    for line in file_read.readlines():
        print(line)Copy the code

When called:

  1. withThe statement is temporarily savedReadFileOf the class__exit__methods
  2. And then callReadFileOf the class__enter__methods
  3. __enter__Method opens the file and returns the result towithstatements
  4. The result of the previous step is passed tofile_readparameter
  5. inwithStatement of thefile_readParameter to operate, read each line
  6. After reading,withTemporary before the statement is called__exit__methods
  7. __exit__Method closes the file

Note that in the __exit__ method, we close the file but return True, so the error is not thrown by the with statement. Otherwise, the with statement throws a corresponding error.