Python logging module details and applications

Summary of the log

Baidu Encyclopedia log overview:

Windows network operating systems are designed to have various log files, such as application logs, security logs, system logs, Scheduler service logs, FTP logs, WWW logs, DNS server logs, and so on, depending on which services are enabled on your system. When we do something on the system, these log files usually record the relevant content of our operation, which is very useful to the system security staff. For example, when someone probes the system for IPC, the system will quickly record the IP address, time, and user name used in the probe in the security log. After FTP probe, the system will record the IP address, time, and user name used in the probe in the FTP log.

Log in my image:

Viewing logs is the best way for developers to obtain daily information, check exceptions and find problems. Log records are usually marked with abnormal causes, occurrence time, specific error lines and other information, which greatly saves our troubleshooting time and virtually improves coding efficiency.

Classification of log

We can classify by output terminal or log level. Output terminal refers to the log in the console output display and log into the file; Log levels include Debug, Info, WARNING, ERROR, and CRITICAL.

Python 的 logging

Logging provides a convenient set of logging functions: debug(), info(), Warning (), error(), and critical(). Logging functions are named according to the level or severity of the events they are used to track. The standard levels and their applicability are described as follows (in ascending order of severity) :

The numeric values for each level are CRITICAL: 50, ERROR: 40, WARNING: 30, INFO: 20, DEBUG: 10, and NOTSET: 0. The default log level in Python is WARNING. DEBUG and INFO logs will not be displayed. Change the Settings in logging.

Log output

Output to console

Use logging to print logs on the console. Here we use the Pycharm editor to observe:

import logging


logging.debug('Cui Qingcai | Jing Mi, Wei Shidong | Quinn')
logging.warning('Invite you to follow the wechat official number [Attack Coder]')
logging.info('Coding and making progress together with big Guy')

Copy the code

Is there any way to change the default log level?

BasicConfig is provided in Logging to allow users to adjust the default logging level. We can change the above code to:

import logging

logging.basicConfig(level=logging.DEBUG)
logging.debug('Cui Qingcai | Jing Mi, Wei Shidong | Quinn')
logging.warning('Invite you to follow the wechat official number [Attack Coder]')
logging.info('Coding and making progress together with big Guy')
Copy the code

Set the level of the level parameter in basicConfig.

Think: can DEBUG messages be displayed if the level is logging.info?

Save to file

Now that you’ve shown how to output log content on the console and set the log level freely, let’s see how to save logs to a file. Still powerful basicConfig, let’s change the above code to:

import logging

logging.basicConfig(level=logging.DEBUG, filename='coder.log', filemode='a')
logging.debug('Cui Qingcai | Jing Mi, Wei Shidong | Quinn')
logging.warning('Invite you to follow the wechat official number [Attack Coder]')
logging.info('Coding and making progress together with big Guy')

Copy the code

With a simple code setup, we have finished exporting the log files to the console and files. What if it’s displayed on the console and saved to a file?

Powerful logging

The module-level logging functions provided by Logging encapsulate the logging logging system-related classes

The Logging module provides two ways to log:

• Use module-level functions provided by Logging
• Four components of the Logging system are used

Loggers, Handlers, filters and Formatters are the four main components. The following diagram illustrates their functions simply:

Four major components

Here are the classes associated with logging components: Logger, Handler, Filter, and Formatter.

The Logger class

A Logger object has three jobs to do:

1) Expose application code to several methods that allow applications to log messages at run time; 2) Determine which logs should be processed based on log severity level (the default filtering facility) or filter object; 3) Log messages are sent to all interested log handlers.Copy the code

The most common methods for Logger objects fall into two categories: configuration methods and message sending methods

The most common configuration methods are as follows:

Note about the logger.setlevel () method:

The lowest level is DEBUG and the highest level is CRITICAL. For example, if setLevel(logging.info) is set to INFO, only INFO, WARNING, ERROR, and CRITICAL logs will be processed. DEBUG messages will be ignored or discarded.

Once the Logger object is configured, you can use the following methods to create logging:

The logging.getLogger() method has an optional parameter name that represents the name identifier of the logger to be returned, or ‘root’ if this parameter is not provided. Calling the getLogger() method multiple times with the same name parameter value returns a reference to the same Logger object.

Logger hierarchy and effective levels Logger names are a hierarchy divided by '.', each logger after '.' is the children of the logger before '.' for example, there is a logger named Foo, The other names are foo.bar, foo.bar.baz, and foo.bam are all descendants of foo. Logger has a concept of an effective level. If no level is explicitly set on a logger, then the logger is the level that uses its parent. If its parent does not have a level set, it continues to look up the valid level of the parent's parent, and so on, until it finds an ancestor that has a level set. Note that root Logger always has an explicit level setting (WARNING by default). When deciding whether to handle an event that has already occurred, logger's effective level is used to determine whether to pass the event to the Handlers of that logger for processing. After child Loggers have finished processing log messages, they default to the handlers associated with their loGGERS ancestors. Therefore, rather than defining and configuring Handlers for all loggers used in an application, it is sufficient to configure handlers for a top-level Logger and then create child Loggers as needed. We can also turn off this propagation mechanism by setting the propagate attribute for a Logger to False.Copy the code

Handler

The Handler object is used (based on the level of the log message) to distribute the message to the location (file, network, mail, etc.) specified by the Handler. Logger objects can add zero or more handler objects to themselves using the addHandler() method. For example, an application might want to implement the following logging requirements:

1) Send all logs to a log file; 2) Send all logs with severity greater than or equal to Error to STDOUT (standard output); 3) Send all critical logs to a single email address. This scenario requires three different Handlers, each complex to send a particular severity level log to a particular location.Copy the code

There are very few methods in a handler that the application developer should care about. For application developers using built-in Handler objects, the only handler methods that seem relevant are the following configuration methods:

Formater

The Formater object is used to configure the final order, structure, and content of log information. Unlike the logging.handler base class, application code can directly instantiate the Formatter class. Alternatively, if your application needs some special processing behavior, you can implement a subclass of Formatter to do it.

The Formatter class constructor is defined as follows:

logging.Formatter.__init__(fmt=None, datefmt=None, style=The '%')
Copy the code

The constructor accepts three optional arguments:

• FMT: Specifies the message formatting string. If this parameter is not specified, the original value of message is used by default
• Datefmt: specifies the date format character string. If this parameter is not specified, “%Y-%m-%d %H:% m :%S” is used by default.
• Style: Python 3.2 new argument, can be ‘%’, ‘{‘ and ‘$’, if not specified default ‘%’

Filter

The Filter can be used by handlers and Loggers to perform more fine-grained and complex filtering than level. Filter is a Filter base class that only allows log events at a logger level to pass the Filter. The class is defined as follows:

class logging.Filter(name=' ')
    filter(record)
Copy the code

For example, if a filter instance passes the name parameter ‘A.B’, the filter instance will only allow logs generated by loGGERS with names like the following: ‘a.’, ‘a., C’, ‘A.B.C.D’, ‘A.B.D’, and the name is’ a. B ‘, ‘B.A.B loggers produce logs will be filtered out. If the value of name is an empty string, all log events are allowed to pass the filtering.

The filter method is used to control whether the passed record can pass filtering. If the return value of this method is 0, it indicates that the record cannot pass filtering. If the return value is non-0, it indicates that the record can pass filtering.

Note: You can also change the record within the Filter (Record) method if necessary, such as adding, deleting, or modifying properties. We can also use the Filter to do some statistics, such as counting the number of records processed by a particular Logger or handler.Copy the code

We practice

So much for copying and pasting, now it’s time to get your hands dirty.

Now I need to both export logs to the console and save logs to a file. What do I do?

Using what we have just learned, we can imagine:

It looks like it’s not hard, it’s easy, but is it?

In actual work or applications, we may also need to specify the file storage path, use random number as the log file name, display the specific code lines of information output, log output date and log writing mode, etc. Think again:

import os
import logging
import uuid
from logging import Handler, FileHandler, StreamHandler


class PathFileHandler(FileHandler):
    def __init__(self, path, filename, mode='a', encoding=None, delay=False):

        filename = os.fspath(filename)
        if not os.path.exists(path):
            os.mkdir(path)
        self.baseFilename = os.path.join(path, filename)
        self.mode = mode
        self.encoding = encoding
        self.delay = delay
        if delay:
            Handler.__init__(self)
            self.stream = None
        else:
            StreamHandler.__init__(self, self._open())


class Loggers(object):
    Log level relational mapping
    level_relations = {
        'debug': logging.DEBUG, 'info': logging.INFO, 'warning': logging.WARNING,
        'error': logging.ERROR, 'critical': logging.CRITICAL
    }

    def __init__(self, filename='{uid}.log'.format(uid=uuid.uuid4()), level='info', log_dir='log',
                 fmt='%(asctime)s - %(filename)s[line:%(lineno)d] - %(levelname)s: %(message)s') : self.logger = logging.getLogger(filename) abspath = os.path.dirname(os.path.abspath(__file__)) self.directory = os.path.join(abspath, log_dir) format_str = logging.Formatter(fmt)Set the log format
        self.logger.setLevel(self.level_relations.get(level))  # Set the log level
        stream_handler = logging.StreamHandler()  # Output to screen
        stream_handler.setFormatter(format_str)
        file_handler = PathFileHandler(path=self.directory, filename=filename, mode='a')
        file_handler.setFormatter(format_str)
        self.logger.addHandler(stream_handler)
        self.logger.addHandler(file_handler)


if __name__ == "__main__":
    txt = "Follow the public number [attack Coder], reply" log code "to get the complete code and flow chart in the article"
    log = Loggers(level='debug')
    log.logger.info(4)
    log.logger.info(5)
    log.logger.info(txt)

Copy the code

After the file is saved, the running result is as follows:

The log is actually printed on the console. Let’s see if the specified files and folders are generated in the directory:

When the file is opened, you can see the output:

What is the right way to learn

Do you step by step look at the article introduction, waiting for the blogger’s conclusion?

Do you take the code and run it?

No, it should be reading the article while taking sample code and thinking and researching what could be improved and what could be better designed. If you need the sample code and flow chart used in the article, then follow the wechat official number [Attack Coder] and reply “Log code” to get the complete code and flow chart in the article. After all, learning is hard work.

References:

Cloud travel Taoist blog

Nancy05 post