1. Confusing operations

This section compares some of Python’s confounding operations.

1.1 With and without reinserted random sampling

Import random random. Choices (seq, k=1) # select random. Sample (seq, k) # select randomCopy the code

1.2 Arguments of the lambda function

Func = lambda y: x + y # x values are bound when the function is runCopy the code

1.3 copy and deepcopy

Import copy y = copy.copy(x) # copy only the top layer y = copy.deepCopy (x) # copy all nested partsCopy the code

When replication is combined with variable aliases, it can be confusing:

a = [1, 2, [3, 4]]

# Alias.
b_alias = a  
assert b_alias == a and b_alias is a

# Shallow copy.
b_shallow_copy = a[:]  
assert b_shallow_copy == a and b_shallow_copy is not a and b_shallow_copy[2] is a[2]

# Deep copy.
import copy
b_deep_copy = copy.deepcopy(a)  
assert b_deep_copy == a and b_deep_copy is not a and b_deep_copy[2] is not a[2]
Copy the code

Changes to an alias affect the original variable. Elements in a (shallow) copy are aliases of elements in the original list, whereas deep copies are replicated recursively and changes to deep copies do not affect the original variable.

1.4 = = and is

Does x == y # have the same value? Does x is y # refer to the same objectCopy the code

1.5 Judgment Type

Type (a) == int # Ignores polymorphisms in object-oriented design isinstance(a, int) # considers polymorphisms in object-oriented designCopy the code

1.6 String Search

str.find(sub, start=None, end=None); str.rfind(...) STR. Index (sub, start=None, end=None); str.rindex(...) Raise ValueError if not foundCopy the code

1.7 List Backward index

This is just a matter of habit. For forward indexes, the index starts at 0. For reverse indexes, you can use ~ if you want to start at 0.

print(a[-1], a[-2], a[-3])
print(a[~0], a[~1], a[~2])
Copy the code

2. C/C++ User Guide

Many Python users have migrated from C/C++. There are some differences between the two languages in terms of syntax, code style, and so on, which this section briefly describes.

2.1 Large numbers and Small numbers

The C/C++ convention is to define a large number. Python has inf and -INF:

a = float('inf')
b = float('-inf')
Copy the code

2.2 Boolean value

While THE C/C++ convention is to use 0 and non-0 values for True and False, Python recommends directly using True and False for booleans.

a = True
b = False
Copy the code

2.3 Null

The C/C++ convention for null Pointers is if (a) and if (! A). Python defines None as:

if x is None:
    pass
Copy the code

If not x is used, all other objects (such as strings of length 0, lists, tuples, dictionaries, etc.) will be treated as False.

2.4 exchange value

The C/C++ convention is to define a temporary variable that is used to exchange values. With Python’s Tuple operation, you can do it all in one step.

a, b = b, a
Copy the code

2.5 compare

The C/C++ convention is to use two conditions. You can do this in one step with Python.


if 0 < a < 5:
    pass
Copy the code

2.6 Set and Get for class members

The C/C++ convention is to make class members private and access their values through a series of Set and Get functions. While it is possible to Set the corresponding Set and Get functions via @property, @setter, and @deleter in Python, we should avoid unnecessary abstraction, which can be 4-5 times slower than direct access.

2.7 Input and output parameters of the function

The C/C++ convention is to list the input and output parameters as the parameters of the function, and change the value of the output parameters through Pointers. The return value of the function is the execution status, and the function caller checks the return value to determine whether the execution is successful. In Python, there is no need for the function caller to check the return value. When a function encounters a special case, it simply throws an exception.

2.8 read the file

Python is much easier to read files than C/C++. The opened file is an iterable that returns one line at a time.

With open(file_path, 'rt', encoding=' utF-8 ') as f: for line in f: print(line) #Copy the code

2.9 Splicing File paths

The C/C++ convention is to concatenate paths directly with +, which is error-prone. Python os.path.join automatically adds/or \ delimiters between paths depending on the operating system:

import os
os.path.join('usr', 'lib', 'local')
Copy the code

2.10 Parsing command-line Options

Argv can be used to parse command-line selections directly in Python as in C/C++, but it is more convenient and powerful to use the ArgumentParser tool under ArgParse.

2.11 Invoking External Commands

While it is possible in Python to call external commands directly using os.system, as in C/C++, subprocess.check_output is optional to execute the Shell and also to obtain external command execution results.

Import subprocess # If the external command returns a non-zero value, The thrown subprocess. CalledProcessError abnormal result = subprocess. Check_output ([' CMD ', 'arg1, Result = subprocess.check_output([' CMD ', 'arg1', 'arg2'], Stderr = subprocess.stdout).decode(' utF-8 ') Can use shlex. Quote () the parameter double quotation marks result = subprocess. Check_output (' grep python | wc > out ', shell = True). The decode (' utf-8)Copy the code

2.12 Do not duplicate the wheel

Don’t reinvent the wheel. Python is called batteries included because Python provides solutions to many common problems.

3. Common tools

3.1 Reading and Writing a CSV file

Import CSV # With open(name, 'rt', encoding=' UTF-8 ', newline=' ") as f: # newline= "for row in csv.reader(f): Print (row[0], row[1]) print(row[0], row[1]) F_csv = csv.writer(f) f_csv.writerow(['symbol', 'change']) # for row in csv.DictReader(f): print(row['symbol'], row['change']) with open(name, mode='wt') as f: header = ['symbol', 'change'] f_csv = csv.DictWriter(f, header) f_csv.writeheader() f_csv.writerow({'symbol': xx, 'change': xx})Copy the code

_Csv. Error: Field LARGER than field limit (131072)

import sys
csv.field_size_limit(sys.maxsize)
Copy the code

CSV can also read \t split data

f = csv.reader(f, delimiter='\t')
Copy the code

3.2 Iterator tools

Itertools defines a number of iterator tools, such as the subsequence tool:

import itertools itertools.islice(iterable, start=None, stop, step=None) # islice('ABCDEF', 2, None) -> C, D, E, F itertools. filterFalse (predicate, iterable) # filterFalse (lambda x: X < 5, [1, 4, 6, 4, 1]) -> 6 itertools. Takewhile (predicate, iterable) # stop iterating when the predicate is False # takewhile(lambda x: X < 5, [1, 4, 6, 4, 1]) -> 1, 4 itertools.dropwhile(predicate, iterable) # dropwhile(lambda x: x < 5, [1, 4, 6, 4, 1]) -> 6, 4, 1 itertools.compress(iterable, Compress ('ABCDEF', [1, 0, 1, 0, 1, 1]) -> A, C, E, FCopy the code

Sequence sorting:

Sorted (iterable, key=None, reverse=False) itertools.groupby(iterable, key=None) 3. Sorted by(sorted([1, 4, 6, 4, 1])) -> (1, iter1), (4, iter4) Iter6) itertools. Permutations (iterable, r=None) Tuple # permutations('ABCD', 2) -> AB, AC, AD, BA, BC, BD, CA, CB, CD, DA, DB, DC Tuple itertools.combinations_with_replacement(...) # combinations('ABCD', 2) -> AB, AC, AD, BC, BD, CDCopy the code

Multiple sequence merging:

Itertools. chain(*iterables) # chain('ABC', 'DEF') -> A, B, C, D, E, F import heapq heapq.merge(*iterables) Key =None, reverse=False) # merge('ABF', 'CDE') -> A, B, C, D, E, F zip(*iterables) # stop when the shortest sequence is used up Itertools. zip_longest(*iterables, fillValue =None) # if the longest sequence is exhausted, the result can be consumed only onceCopy the code

3.3 the counter

A counter counts the number of occurrences of each element in an iterable.

Import collections # create collections.counter (iterable) # collections.counter [key] # key If n is None, Returns all elements in the collections. Counter. Most_common (n = None) # insert/update the collections. Counter. Update (iterable counter1 + counter2); Collections.counter (list1) == collections.counter (list2)Copy the code

3.4 Dict with default Values

When accessing a nonexistent Key, DefaultDict sets it to some default value.

Import collections Collections.defaultdict (type) # When accessing dict[key] for the first time, type is called without arguments, giving dict[key] an initial valueCopy the code

3.5 the orderly Dict

Import collections collections.ordereddict (items=None) # Preserve the original insertion order during iterationCopy the code

4. High-performance programming and debugging

4.1 Error and Warning Information Is Displayed

Output information to standard error

import sys
sys.stderr.write('')
Copy the code

Output Warning message

Import warnings warnings. Warn (message, category=UserWarning) RuntimeWarning, ResourceWarning, FutureWarningCopy the code

Controls the output of warning messages

Simplefilter ('always') $python -w ignore # Ignore all warnings, $python -w error # Convert all warnings to exceptions, equivalent to setting warnings. Simplefilter ('error')Copy the code

4.2 Testing in code

Sometimes, for debugging purposes, we want to add some code to our code, usually some print statements, which can be written as:

# Debug in the code if __debug__: passCopy the code

Once debugging is complete, this part of the code is ignored by executing the -o option on the command line:

$ python -0 main.py
Copy the code

4.3 Code style Check

Pylint allows for quite a bit of code style and syntax checking to catch errors before they are run

pylint main.py
Copy the code

4.4 Code Time

Take the test

$ python -m cProfile main.py
Copy the code

Time consuming to test a block of code

From contextlib import contextManager from time import perf_counter @contextManager def timeblock(label): tic = perf_counter() try: yield finally: toc = perf_counter() print('%s : %s' % (label, toc-tic) # Timeblock ('counting'): passCopy the code

Some principles for code time optimization

  • Focus on optimizing where performance bottlenecks occur, not the entire code.
  • Avoid using global variables. Local variables are faster to find than global variables, and running code defined for global variables in functions is typically 15 to 30 percent faster.
  • Avoid using.access properties. It is faster to use from Module import name to put self.member of frequently visited classes into a local variable.
  • Use built-in data structures whenever possible. STR, list, set, dict, etc. are implemented by C and run quickly.
  • Avoid creating unnecessary intermediate variables, and copy.deepCopy ().
  • String concatenation such as a + ‘:’ + b + ‘:’ + c creates a lot of useless intermediate variables. ‘:’,join([a, b, c]) is much more efficient. Also consider whether string concatenation is necessary, for example print(‘:’.join([a, b, c])) is less efficient than print(a, b, c, sep=’:’).

5. Other Python tricks

5.1 argmin and argmax

items = [2, 1, 3, 4]
argmin = min(range(len(items)), key=items.__getitem__)
Copy the code

Argmax similarly.

5.2 Transposed two-dimensional list

A = [['a11', 'a12'], ['a21', 'a22'], ['a31', 'a32']]
A_transpose = list(zip(*A))  # list of tuple
A_transpose = list(list(col) for col in zip(*A))  # list of list
Copy the code

5.3 Expanding a one-dimensional list to a two-dimensional list

A = [1, 2, 3, 4, 5, 6]

# Preferred.
list(zip(*[iter(A)] * 2))
Copy the code

Author: Beijiang patriotic