Most Python applications use both the local standard library and the third-party library, which not only allows us to focus on real business development, but also allows us to learn more valuable programming and development ideas. There is a famous saying in program development called:

Don’t re-invent the wheel

Simplicity says don’t do the same thing. Since someone has done what you need to do, you should use it where appropriate to avoid unnecessary time wasters.

Python libraries can be divided into two broad categories: standard libraries and tripartite libraries. Most of the standard libraries are built into files when you download the corresponding Python version, and can be called directly from the program and downloaded from the Internet without using the PIP package management tool. The tripartite library is a function library or by-product gradually derived from the actual development of other programmers in order to better achieve a certain goal, and then put it on the Internet for us to share.

Both standard and tripartite libraries are valuable for us to learn. Here are some examples of the standard libraries that might be used in real development (tripartite Libraries, part 2) to impress you.

The standard library

sys

System-specific parameters and functions. The SYS library is used to access and modify system-specific information, such as viewing python versions, system environment variables, module information, and Python interpreter information.

os

Operating system interface module. This library provides access to operating system-specific dependencies, such as input/output operations, read/write operations, operating system exception error messages, process thread management, file management, schedulers, and more.

re

Regular expression manipulation. This library is my favorite and often used library, when processing a large number of strings using regular expressions is the fastest and most effective way, but regular expressions have a high learning curve, interested friends can visit this website to learn.

math

Library of mathematical functions. The Math library provides access to mathematical functions defined by the C language standard, Embed theory, Power and logarithmic functions, Trigonometric functions, constant PI (π) and natural constant (e).

random

Generate pseudo-random numbers.

Pseudo random Numbers and random Numbers (true random number) is different execution environment, random number is through the physical process in the real world practice, concluded that the pseudo random number is the specific algorithm by computer generated, so the process is predictable and regular, just cycle is long, and not with the real scene is appropriate.

The random library provides the generation of random numbers, which can simulate the random number selection, random lottery and so on in the real world.

logging

Logging tools. This library provides the logging of the application program and library functions. In daily development, we often need to print out the running status of the current program through the log, and check the possible stack exceptions and error information in real time.

Default log printing level (customizable) :

level value
critical 50
error 40
abnormal 30
information 20
debugging 10
Is not set 0

json

Json encoding and decoder. Json library provides support for JSON data. In daily development, we need to serialize and deserialize the transmitted data JSON to ensure the integrity and effectiveness of data. Serialization and deserialization are actually the process of encoding and decoding.

pickle

Python object serialization library. The pickle library supports serialization and deserialization of Python objects. When we need to save the processed objects to a file or database, we can serialize them to binary data for better storage.

shelve

Python object persistence. Simple data storage solution.

socket

Underlying network interface. The Socket library provides the standard BSD (Berkeley Socket) SOCKET API, which can be used for network communication by accessing the socket interface of the underlying operating system.

datetime

Base date and time type library. The library provides a variety of simple and complex ways to deal with dates and times. We use time to measure time consumption and complexity on a daily basis. We also need to elaborate on the creation and modification times of stores.

hashlib

Security hash and message digest. Algorithm is actually to encrypt some data (irreversible encryption algorithm), because the encrypted data can not be cracked, so it can prevent tampering. The common digest algorithms include MD5 and SHA1. Generally, MD5 is used to encrypt the user password to prevent it from being easily cracked after embezzlement. SHA1 is similar to MD5, but it has a longer length and is more secure. However, the complexity of the algorithm is usually accompanied by the consumption of storage space and time. For characters longer than SHA1, there are SHA224, SHA256, SHA384, and SHA512.

Everyone knows that collisions are inevitable regardless of the length of the character generated by the algorithm, so it depends on the situation.

configparser

Configure the file resolver. The ConfigParser library makes it easy to customize configuration files, and provides global access by parsing the information in the configuration file.

urllib

URL processing module. The URllib library integrates various modules that handle URLs:

  • Urllib. request access and readURLdata
  • Urllib. Contains the errorurllib.requestThrown exception
  • Urllib. Parse parsingURLdata
  • Urllib. Robotparser parsingrobots.txtfile

The URllib library provides good support for accessing the network, providing access and processing of data, uploading and downloading of files, logging cookies and sessions, and more.

itertools

A function that creates iterators for efficient loops. The IterTools library is also used a lot. When we want to do for-in on something, we need to process it into an iterable before we can iterate.

collections

Container data type library. The Collections library provides support for all container data types, including dict, list, set, and tuple. We can use this library to operate on different data types. Common function methods include:

  • Namedtuple () creates a factory function that subclasses named tuples
  • A deque is a container similar to a list, enabling quick appends and pops at both ends.
  • A ChainMap is like a dict container class that aggregates multiple mappings into a single view
  • A subclass of the Counter dictionary that provides counting of hashable objects
  • Subclasses of the OrderedDict dictionary, preserving the order in which they were added
  • Defaultdict, a subclass of the dictionary, provides a factory function that provides a default value for dictionary queries
  • UserDict encapsulates dictionary objects, simplifying dictionary subclassing
  • UserList encapsulates list objects and simplifies list subclassing
  • UserString encapsulates list objects and simplifies string subclassing

functools

Higher-order functions and operations on callable objects. This library mainly calls higher-order functions and is a supplement to regular functions. The library currently contains the following functions:

  • cmp_to_key
  • lru_cache
  • total_ordering
  • partial
  • partialmethod
  • reduce
  • singledispatch
  • update_wrapper
  • wraps

threading

Thread parallel library. The threading library supports both threading and multithreading operations. The threading library also provides synchronization locks for data that can only be processed by one thread at a time.

On CPython interpreters, only one thread can execute at a time because the GIL (global interpreter lock) mechanism is designed to be thread-safe, which prevents multithreading from taking advantage of the multicore nature of computers.

multiprocessing

Process parallel library. The Multiprocessing library is similar to the Threading library, except that the process library can create child processes to bypass the GIL to compensate for thread inventory and exploit the multi-core nature of the computer.

timeit

Measure the execution time of small code snippets. This library is mainly used to calculate the elapsed time of running code, and supports various ways to pass in parameters.

atexit

Exit the processor. This library can be used when processing a function that requires immediate exit.

abc

Abstract base classes. The ABC library defines an abstract base class so that other classes can derive new classes. For example, the Collections container library has the collections. ABC class derived from this, which can be further implemented.

asyncio

Asynchronous I/O library. Asyncio library is a concurrent library written with async/await keyword, which provides basic functions for multiple asynchronous frameworks and can realize high performance network, Web server, database connection and distributed task queue.

The Asyncio library in asynchronous development provides two classes of apis, high-level apis:

  • Coroutines and Tasks
  • Streams
  • Synchronization Primitives
  • Subprocesses
  • Queues
  • Exceptions

And low level apis:

  • Event Loop
  • Futures
  • Transports and Protocols
  • Policies
  • Platform Support

base64

Data coding library. The Base64 library is used to encode raw binary data for easy dissemination over the Internet.

For example, when we need to use JSON data to transmit an image, we need to encode the binary data of the image first (JSON cannot carry binary data) and save it in value. When the other party gets the JSON data, it can decode it with the same algorithm to obtain binary image data.

The Base64 library provides a variety of encoding algorithms, the most famous of which is base64 named after him. The encoding method of Base64 is to encode 64 states of binary data 000000 to 111111 into 0-9 digits, A-Z, A-Z letters, plus signs and/signs in a group of six bits. When binary data is less than 6 bits, it is processed by means of complement and tag, see here.

In addition to Base64, the Base64 library provides other encoding algorithms, such as Base16, Base32, Ascii85, and Base85. It is important to note that Base64 is not strictly a form of encryption, but rather an encoding transformation for converting to visible strings.

concurrent.futures

Start parallel tasks. The concurrent.futures library provides a high-level interface to implement asynchronous calls in two ways: ThreadPoolExecutor for threads and ProcessPoolExecutor for processes. As the name suggests, Both are implemented from the Executor abstract class.

copy

Shallow and deep copy operations. The copy library provides a copy of an object. As we all know, you can’t make a copy of an object by simply passing a value to create a new variable, because the memory that the new variable points to is still the original object itself, so any operation on the new variable will change the original object. The copy library, then, provides a variety of ways to make copies of objects, creating a new memory space to hold them without any interference with the original object.

csv

CSV (Separated Values) file read and write library. This library supports storing tabular data (numeric and text) in plain text.

operator

Standard operators replace libraries. This library uses Python’s own operators as valid functions. For example, the expression x+y can be represented by the operator.add(x, y) function; For example, the expression a*b can be represented by the function operator.mul(a, b), etc.

enum

Enumeration library. The enum library supports the creation of enumerated classes to store a large number of immutable constants of the same type for other functions to call. The resulting enumeration class is an iterable, so you can use for-in to enumerate all constants.

heapq

Heap queue algorithm. This module provides an implementation of the heap queue algorithm, also known as the priority queue algorithm. Each element in the priority queue has its own priority, and the element with the highest priority gets served first. So when we require the maximum/minimum value of the first n, we can use this algorithm to achieve, and the corresponding function is also provided in heAPQ library.

http

The HTTP module. The HTTP module is a package that collects multiple modules that handle hypertext Transfer protocols:

  • Http. client A low-level HTTP client used for high-level URL accessurllib.request
  • Http. server Basic HTTP service class based on socketServer
  • HTTP. Cookies Cookies status management tool
  • The HTTP.cookiejar provides persistence of cookies

The HTTP module defines the HTTP status code and associated messages through the HTTP. HTTPStatus enumeration.

Profile, pstats

Performance analysis tool. Profile module provides profile and cProfile two different implementation of the performance analysis tools, can be used to describe the execution time and frequency of each part of the program, statistics can be saved and used by the PSTATS module.

ssl

TLS/SSL (Transport Security Protocol). This module provides support for security protocols. Through application context, TLS (Transport Layer Security Protocol) or its predecessor SSL (Secure Sockets Layer) can support security protocols to ensure the security and data integrity of Internet communications. Generally, HTTPS supports TLS/SSL encryption.

unitest

Unit testing framework. Unitest libraries are often used for unit testing, and they have similar features and functions inspired by JUnit and other mainstream testing libraries.

uuid

UUID library. Uuid1 (), UUid3 (), UUid4 (), and UUid5 () for uuID1, 3, 4, and 5. Note that uuid1() and UUid4 () can be used to generate random strings, but uuid1() has privacy risks because it contains the network address of the user accessing the computer, while UUid4 () is generated from random characters.

Uuid1 () is not necessarily a safe way to generate uuid1(). There is no synchronization lock when two processes are processing uuid1() at the same time, so there is no guarantee that two uuids will not be generated.

conclusion

Is the default language pack in the existence of the standard library, although most of the basic functions have been implemented, which means that we don’t need to repeat the wheels, but we still need to take the time to pay attention to and understand one of the program design thinking, learning about how open source recognize will play solid foundation, the development of the upper application for us to establish the foundation.

Note:

  • This article continues to be updated, and more valuable libraries are welcome.
  • Python3 Chinese wiki and English wiki.