Basic usage of the Logging module in Python

This article was originally published in: Walker AI

When developing a project, we can’t print all the information in the console. We can use the Logging API provided by the Python standard library to handle this. Compared to print, the Logging module provides many powerful and flexible features. For example, you can set different log levels to output only important information instead of displaying a lot of mode information. Print prints all the information to standard output, seriously affecting the developer’s ability to view other data from standard output, while logging allows the developer to decide where and how to print the information.

1. Basic logging module information

1.1 Log Levels

The logging function defines five logging levels by default. It allows developers to customize logging levels, but it is not recommended.

Note: The log level increases from top to bottom DEBUG < INFO < WARNING < ERROR < CRITICAL, while the log information decreases in turn. When a log level is specified, logs whose value is greater than or equal to this level are recorded. Logs whose value is smaller than this level are discarded. The default log level is WARNING. Logs are traced only when the log level is set to WARNING or higher.

1.2 Log field letters and formats

A log corresponds to the occurrence of an event, and an event usually contains the following contents:

• Time of event
• Location of event
• The severity of the incident
• Log level and event content

Note: When a log is output, the log content and log level need to be specified explicitly by the developer. For other field information, you only need to display it in the log.

2. Usage of the Logging module

2.1 Two Log Recording modes

(1) Using the module-level functions provided by Logging (2) Using the four components of the logging system

2.2 Module-level functions provided by Logging are described in detail

(1) Function description:

BasicConfig (**kwargs) is used to specify “log level to be recorded”, “log format”, “log output location”, “log file opening mode” and other information, the other several are used for logging at various levels of the function.

(2) Examples:

import logging
def ex_logging() :
    logging.debug("Log level: DEBUG")
	logging.info("Log level: INFO")
	logging.warning("Log level: WARNING")
  	logging.error("Log level: ERROR")
 	logging.critical("Log level: CRITICAL")
ex_logging()
Copy the code

(3) Actual results:

WARNING:root: log level: WARNING ERROR:root: log level: ERROR CRITICAL:root: log level: CRITICALCopy the code

Note: The default log level is WARNING. Therefore, only the logs whose level is higher than or equal to WARNING are displayed in the actual result.

2.3 Using the four components of the Logging system

Component Description:

2.4 Logger – Logger

(1) Logger is a tree hierarchy. You must create Logger instances before using debug, INFO, Warning, error, and critical interfaces.

(2) Creation method:

logger = logging.getLogger(logger_name)
Copy the code

(3) After creating Logger instance, you can use the following methods to set the log level and add Handler:

logger.setLevel(logging.ERROR) # set the log level to ERROR, that is, only logs whose log level is greater than or equal to ERROR will be output
logger.addHandler(handler_name) Add a handler to the Logger instance
logger.removeHandler(handler_name) Remove a handler for the Logger instance
Copy the code

2.5 Processor – Handler

There are many different types of handlers, including StreamHandler, FileHandler, and NullHandler.

(2) Creation method:

# StreamHandler creation method
sh = logging.StreamHandler(stream=None)
Create FileHandler
# fh = logging.FileHandler(filename, mode='a', encoding=None, delay=False)
# NullHandler: The NullHandler class is in the core logging package and does not do any formatting or output. It is essentially a "do nothing" handler used by library developers.
Copy the code

(3) After creating a StreamHandler, you can set the log level, set the Formatter, and add or remove the Filter by using the following methods:

ch.setLevel(logging.WARN) # Specify a log level. Logs lower than WARN are ignored
ch.setFormatter(formatter_name) Set a formatter
ch.addFilter(filter_name) # Add a filter. You can add more filters
ch.removeFilter(filter_name) Delete a filter
Copy the code

2.6 filter-filters

(1) Filter base class, which only allows log events at a logger level to pass filtering.

(2) A log message is output through the following filters: loger-level filtering, loger-level filtering, processor-level filtering, and processor filtering.

(3) Creation method:

filter = logging.Filter(name=' ')
Copy the code

2.7 Formatter-formatter

(1) Use Formatter to set the last rule, structure, and content of log information. The default time format is %Y-% M -%d %H:% m :%S.

(2) Creation method:

formatter = logging.Formatter(fmt=None, datefmt=None)
# FMT is the format string for the message and datefmt is the date string. If FMT is not specified, '%(message)s' is used. If datefMT is not specified, the ISO8601 date format is used.
Copy the code

2.8 Relationships between Components

(1) The logger needs to output the log information to the target location through handler. Different handlers can output the log information to different locations.

(2) Logger can set multiple handlers to output the same log record to different locations.

(3) Each processor can set its own filter to filter logs, so that only interested logs are retained.

(4) Each handler can set its own formatter to output the same log to different places in different formats.

Logger can contain one or more handlers and filters, i.e., LoggerFilter, i.e., Logger is one-to-many with Handler or Fitler. A Logger instance can add multiple handlers, a Handler can add multiple formatters or filters, and the logging level will be inherited.

3. Code implementation and specific application

Logging can be configured in three ways: Loggers, Handlers, and Formatters are created using Python code and their configuration functions are called, creating a log configuration file. FileConfig () is then used to read the file and create a dict containing the configuration information. It is then passed to the dictConfig() function;

3.1 Log Configuration Using Python Code

(1) Python code display:

import logging
import sys
Create logger and set its log level to DEBUG
logger = logging.getLogger("python_config_logger")
logger.setLevel(logging.DEBUG)
Create a stream handler and set its log level to DEBUG
handler = logging.StreamHandler(sys.stdout)
handler.setLevel(logging.DEBUG)
Create a formatter and add it to the handler
formatter = logging.Formatter("%(asctime)s - %(name)s - %(levelname)s - %(message)s")
handler.setFormatter(formatter)
Add the handler handler created above for logger logger
logger.addHandler(handler)
Print logs to the console
logger.debug('Print log level: DEBUG')
logger.info('Print log level: INFO')
logger.warning('Print log level: Warning')
logger.error('Print log level: error')
logger.critical('Print log level: Critical') 
Copy the code

(2) Actual results:

2021- 02 -24 17:13:44.644- python_config_logger-debug - Prints logs at the DEBUG level2021- 02 -24 17:13:44.644- python_config_logger-info - Prints logs at the INFO level2021- 02 -24 17:13:44.644- python_config_logger-warning - Prints the log level: WARNING2021- 02 -24 17:13:44.645- python_config_logger-error - Prints logs at the ERROR level2021- 02 -24 17:13:44.645- python_config_logger-critical - Logs are printed at the CRITICAL levelCopy the code

3.2 Log configuration using the configuration file and fileConfig() function

(1) Python code display:

import logging.config
Read the log configuration file contents
logging.config.fileConfig('logging.conf')
Create a logger
logger = logging.getLogger('simpleExample')
Print logs to the console
logger.debug('Print log level: DEBUG')
logger.info('Print log level: INFO')
logger.warning('Print log level: Warning')
logger.error('Print log level: error')
logger.critical('Print log level: Critical')
Copy the code

(2) Logging.conf configuration file contents:

[loggers]
keys=root,simpleExample
# test
[handlers]
keys=fileHandler,consoleHandler
[formatters]
keys=simpleFormatter
[logger_root]
level=DEBUG
handlers=fileHandler
[logger_simpleExample]
level=DEBUG
handlers=consoleHandler
qualname=simpleExample
propagate=0
[handler_consoleHandler]
class=StreamHandler
args= (sys.stdout,)
level=DEBUG
formatter=simpleFormatter
[handler_fileHandler]
class=FileHandler
args= ('logging.log'.'a')
level=ERROR
formatter=simpleFormatter
[formatter_simpleFormatter]
format= % (asctime)s- % (name)s- % (levelname)s- % (message)s
datefmt=
Copy the code

(3) Actual results:

2021- 02 -25 1504:50.644- simpleExample - DEBUG - Prints The log level is DEBUG2021- 02 -25 1504:50.646- simpleExample - INFO - print The log level is INFO2021- 02 -25 1504:50.646- simpleExample - warning-Print Log level: WARNING2021- 02 -25 1504:50.646- simpleExample - ERROR - Prints The log level is ERROR2021- 02 -25 1504:50.646- simpleExample - CRITICAL - Prints the log level: CRITICALCopy the code

3.3 Use dict configuration information and dictConfig() function to achieve log configuration

(1) Python code display:

import logging.config
import logging
import yaml

Read the log configuration file contents
with open('logging.yml'.'r') as file_logging:
    dict_conf = yaml.load(file_logging, Loader=yaml.FullLoader)
logging.config.dictConfig(dict_conf)
Create a logger
logger = logging.getLogger('simpleExample')
Print logs to the console
logger.debug('Print log level: DEBUG')
logger.info('Print log level: INFO')
logger.warning('Print log level: Warning')
logger.error('Print log level: error')
logger.critical('Print log level: Critical')  
Copy the code

(2) Logging. yml configuration file contents:

version: 1
formatters:
  simple:
    format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
handlers:
  console:
    class: logging.StreamHandler
    level: DEBUG
    formatter: simple
loggers:
  simpleExample:
    level: DEBUG
    handlers: [console]
    propagate: no
root:
  level: DEBUG
  handlers: [console]

Copy the code

(3) Actual results:

2021- 02 -25 15:14:28.822- simpleExample - DEBUG - Prints The log level is DEBUG2021- 02 -25 15:14:28.822- simpleExample - INFO - print The log level is INFO2021- 02 -25 15:14:28.822- simpleExample - warning-Print Log level: WARNING2021- 02 -25 15:14:28.822- simpleExample - ERROR - Prints The log level is ERROR2021- 02 -25 15:14:28.823- simpleExample - CRITICAL - Prints the log level: CRITICALCopy the code

4. To summarize

This article has only briefly introduced the basic usage of the Logging module in Python, and there are many optimizations to be explored in the future.

PS: more dry technology, pay attention to the public, | xingzhe_ai 】, and walker to discuss together!