Preface Moment:

It’s time to summarize again. I haven’t learned much today

Summary of a wave:

Today we learned the Shutil module, logging module

1. Shutil module

Python officially defines the shutil module as an advanced file manipulation module, which deals with the use of folders and files to copy, delete, and compress files

1) shutil.rmtree(path, ignore_errors=False, onerror=None) : Delete folder

• Path: Must be a folder path
• Ignore_errors: if this parameter is True, errors are ignored and not reported, and vice versa.
• Onerror: callback a function such as func(function, path, excinfo).

Os.path. move indicates that only empty folders or files can be deleted. Use shutil.rmtree to do this.

import shutil

# 1, shutil. Rmtree

shutil.rmtree("test", ignore_errors=True)

Copy the code

2) shutil.move(SRC, DST, copy_function=shutil.copy2) : move the folder

Recursively, move the contents of the SRC folder to the DST folder and return the DST path.

Note: if the DST directory already exists, the SRC folder will be placed in the DST directory folder.

# shutil.move

The test02 folder already exists
shutil.move('test03'.'test02', copy_function=shutil.copy2)
# 'test02/test03'
Copy the code

Shutil. copy2(SRC, DST, *, follow_symlinks=True) : Copies files, but tries to keep file metadata.

• SRC: indicates the path of the original file
• DST: destination path, folder, or file name.
• Follow_symlinks:

# shutil.move

shutil.copy2('test2.txt'.'test3_copy2.txt')   # test3_copy2.txt
shutil.copy('test2.txt'.'test3_copy.txt')   # test3_copy.txt
Copy the code

Shutil. copy(SRC, DST, *, follow_symlinks=True) copies only the file’s data and permissions, and nothing else.

! [shutil. Copy and contrast copy2.c] (/ Users/jiaozhuzhang/Library/Mobile Documents/com apple CloudDocs Mac_Cloud/my blog posts/western childe (blog) / 2021/06 / media/shutil. Copy and copy2.c. PNG)

Shutil. copytree(SRC, DST, symlinks=False, ignore=None, copy_function=copy2, ignore_dangling_symlinks=False, Dirs_exist_ok =False) : Copy folder

• SRC: indicates the path of the original folder
• DST: the folder directory cannot exist, otherwise it is empty, it creates its own directory by default.
• Ignore: sets not to copy some files,

Copy the entire SRC directory into the DST directory. Ignore ignores some files. Shutil.ignore_patterns () supports regular expressions.

# 4, shutil. Copytree
shutil.copytree('test02'.'./test06', ignore=shutil.ignore_patterns("*.py".'test[0-9].txt'))
# './test06'
Copy the code

1.5 shutil. Disk_usage

Shutil. disk_usage(path) : Obtains the usage of the disk where the path resides. Returns a tuple: Total Used Free.

total, used, free = shutil.disk_usage('/')
print(F "Disk size:{round(total/(1024*1024*1024), 1)}GB, used:{round(used/(1024*1024*1024), 1)}GB, remaining:{round(free/(1024*1024*1024), 1)} GB")

# Disk size :465.6 GB, used :230.3 GB, remaining :219.0 GB
Copy the code

1.6 shutil. Make_archive

shutil.make_archive(*base_name*, *format*[, *root_dir*[, *base_dir*[, *verbose*[, *dry_run*[, *owner*[, *group*[, * Logger *]]]]]]]) compress files/folders.

Shutil. unpack_archive(*filename*[, *extract_dir*[, *format*]]) : Decompress the file

Logging logging module

Python’s logging module is extremely important, especially in projects. The functions of logs are as follows:

1. Record user behavior — do data analysis
2. Look for errors in your code.

The log levels are as follows: debug ->info -> Warning -> Error ->crtical

logging.debug("Debugging")
logging.info("Notice of hi!")

logging.warning("Warning")
logging.error("Error")	
logging.critical("Harsh criticism.")

# WARNING: the root: WARNING
# ERROR: root: ERROR
# CRITICAL:root

If you want to display debug and INFO, you need to set the log level.
Copy the code

2.1 Setting a Log Level

logging.basicConfig(**kwargs)
Copy the code

1) Simply set the log level:

logging.basicConfig(level=logging.DEBUG)
logging.debug("Just debug it.")
Just debug it
Copy the code

2) Set the output format of logs:

In general, logs in a project are as follows: time + log level +py name + line + message

logging.basicConfig(level=logging.INFO,
                    format="%(asctime)s - %(name)s - %(levelname)s - %(module)s - %(lineno)d - %(message)s",
                    )
logging.info("This is a notice.")
# 2021-06-13 18:45:52,403 - root-info-day21-shutil and logging modules - 6- This is a notification
Copy the code

More parameters in format can be found below:

The logging.basicconfig () function allows you to change the logging module's default behavior by using the following parameters: filename: creates a FiledHandler with the specified filename, so that the log will be stored in the specified file. Filemode: file opening mode. This parameter is used when filename is specified. The default value is "A" or "w".format: Specifies the log display format used by the handler. Datefmt: Specifies the date and time format. Stream: Creates a StreamHandler with the specified stream. You can specify output to sys.stderr,sys.stdout, or a file (f=open(' test.log ', 'w')), the default is sys.stderr. If both filename and stream are listed, the stream argument is ignored.formatPossible formatting strings in the argument: %(name)s Logger name %(LevelNo)s Digital log level %(LevelName)s Text log level %(pathName)s Full pathname of the module that invokes the log output function, There may not be %(filename)s Name of the module that calls the log output function %(module)s Name of the module that calls the log output function %(funcName)s Name of the function that calls the log output function %(lineno)d Line of code where the statement that calls the log output function resides %(created)f Current time. %(created) specifies the current time. %(ascTime)s Specifies the current time in the string format after the log message is generated using a UNIX standard float. The default format is"2003- 07-0816:49:45.896". After the comma is milliseconds %(thread)d the ID of the thread. There may be no %(threadName)s threadName. There may be no %(process)d process ID. There may be no %(message)s message output by the userCopy the code

3) Store logs to a file and cut logs:

# file_handler is the handle to set the operation to write the log to the file.
file_handler = logging.FileHandler(filename='test.log', mode='at', encoding='utf8')
logging.basicConfig(handlers=[file_handler],
                    format="%(asctime)s - %(name)s - %(levelname)s - %(lineno)s - %(message)s",
                     level=logging.DEBUG,
)
logging.warning("name")

Log 2021-06-13 21:29:30,108 - root - WARNING - 
      
        - name
      
Copy the code

There are two ways to set the cutting log:

1. One way is to cut by time, every number of seconds to cut a log. You can set the number of logs to be retained. If the number exceeds the limit, logs will be automatically deleted.
2. The other method is to cut logs by size. If the log size exceeds a certain value, a log will be cut. You can also set the number of logs to be kept.

Log cutting operation

size_split = logging.handlers.RotatingFileHandler("test01.log", maxBytes=1024*1024, backupCount=5)  Cut up to 1MB and back up 5 copies
time_split = logging.handlers.TimedRotatingFileHandler("test02.log", when='s', interval=10, backupCount=5) 
# when interval Indicates the unit of interval backup. Possible values: s, h, d, etc

logging.basicConfig(
    format='%(asctime)s - %(name)s - %(levelname)s -%(module)s: %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S %p',
    handlers=[size_split,time_split],
    level=logging.ERROR
)

for i in range(1.100000):
    time.sleep(1)
    logging.error('KeyboardInterrupt error %s'%str(i))

Copy the code

4) Set the log can be stored in a file and can be printed to the screen:

Add a logging.streamHandler () handler for output to the screen and a handler for writing to a file.

Log cutting operation
import time
import logging
sh = logging.StreamHandler()
file_handler = logging.FileHandler(filename='test03.log', mode='a', encoding='utf8')
logging.basicConfig(
    format='%(asctime)s - %(name)s - %(levelname)s -%(module)s: %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S %p',
    handlers=[sh, file_handler],
    level=logging.ERROR
)

for i in range(1.10) :# time.sleep(1)
    logging.error('KeyboardInterrupt error %s'%str(i))

Copy the code

Conclusion:

Grasp key points and master them thoroughly.

Reference article:

Docs.python.org/zh-cn/3/lib…

Docs.python.org/zh-cn/3/lib…

www.cnblogs.com/Eva-J/artic…