The introduction

For proper handling of resource management involving exceptions, try/finally code structures are required. Such structures can lead to a cumbersome, unreadable, and unattractive overall code structure, so the with keyword is introduced in Python2.6.

The With as statement is a simplified syntax provided by Pyhton for accessing resources. It ensures that the necessary cleanup operations are performed to release resources regardless of exceptions.


With operation file

For system resources such as files, database connections, and sockets, one of the things an application must do after opening these resources and executing its business logic is to close (release) the resource.

For example, a Python program opens a file, writes something to the file, and then closes the file. What happens if you close the file? Extreme cases can cause Too many open files errors because the system allows you to open a limited number of files.

Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections Can not connect to MySQL Server Too many connections

With open(file) as f = with open(file) as f Let’s take a look at the underlying principles. Let’s see how to close a file properly.


Normal version:

file = "test.txt"

def fun1() :
    """ Regular edition """
    f = open(file, "w")
    f.write("hello python")
    f.close
Copy the code

A potential problem with this is that if an exception occurs during a call to write and subsequent code fails to execute, the close method will not be called properly, so resources will always be released by the program occupiers. So how can you improve the code?


The advanced version:

file = "test.txt"

def fun2() :
    """ Exception Handling """
    try:
        f = open(file, "w")
        f.write("hello python")
    except Exception as e:
        print(e)
    finally:
        f.close()
Copy the code

A modified version of the program tries to catch code where an exception may occur, using a try/finally statement that indicates that if an exception occurs in the try block, subsequent code is not executed and jumps directly to the except block. The finally block will be executed anyway. Therefore, as long as you put close in the finally code, the file is bound to close.


The advanced version:

file = "test.txt"

def fun3() :
    """ with keyword """
    with open(file, "w") as f:
        f.write("hello python")

Copy the code

A more concise and elegant way is to use the with keyword. The return value of the open method is assigned to f. When leaving the with block, the system automatically calls f.close(). With acts as a try/finally statement. So how does it work? Another concept that needs to be covered before we talk about the principle of WITH is the Context Manager.


Context manager

What is context?

Context means different things in different places. In programming, context is simply the environment.

For example, in an APP, when switching screens, you need to save information about which screen you jumped from, so that when you click back, you can jump back correctly. If you don’t save, you can’t jump back correctly.

For example, when a thread or coroutine is switching tasks, how does the program know whether to switch to another task from the beginning or from the middle? Context plays a role, that is, the task itself will save its environment, where it has done, how much it has done, various states will mark and record, thus forming a context environment, so in the switch according to the context of each task, continue to execute, so as to achieve multi-task.


Context manager

Any object whose class implements __enter__() and __exit__() methods can be called a context manager.

Context manager objects can use the with keyword.

file = "test.txt"

def fun3() :
    """ with keyword """
    with open(file, "w") as f:
        # with code block
        f.write("hello python")
        
    print("End of statement with")
Copy the code
  • __enter__(self): goes to the method that the context manager automatically calls in theWith as code blockExecute before execution. If the with statement hasAs clause, the return value of this method will be assigned to the variable after the AS clause; This method can return multiple values, so you can also specify multiple variables after the AS clause (Multiple variables must be enclosed by () to form a tuple).
  • __exit__(self, exc_type, exc_value, exc_traceback): Exits the method automatically invoked by the context manager. This method is executed after the execution of the with as code block. ifThe with as block completed successfully, the program automatically calls this method with all three arguments None, ifWith as code blockbecauseAbort by exception, the program also automatically calls the method, usingsys.exc_infoThe resulting exception information is used as an argument to call the method.


Class-based context manager

We can simulate implementing a file class of our own that implements __enter__() and __exit__() methods.

Implementation principle of the with keyword Context manager

# Implement context manager based on class
class File(object) :

    def __init__(self, filename, mode) :
        self.filename = filename
        self.mode = mode
        self.file = None

    def __enter__(self) :
        When entering the with as statement, the value returned by the with call is used as the variable after as.
        print("__enter__ called")
        self.file = open(self.filename, self.mode)
        return self.file

    def __exit__(self, exc_type, exc_value, exc_traceback) :
        """ called with when leaving with statement """ "
        print("__exit__ called")
        print("exc_type: ", exc_type)
        print("exc_value: ", exc_value)
        print("exc_traceback: ", exc_traceback)
        self.file.close()
        print("File close operation")


def main() :

    with File("test.txt"."w") as f:
        print("With code block")
        f.write("hello python1")
        f.write("hello python2")
        # a = 1 / 0
        f.write("hello python3")

    print("End of statement with")


if __name__ == '__main__':
    main()
Copy the code

The __enter__() method returns the resource object, which is the file object you will open, and the __exit__() method handles some cleanup.

Because the File class implements the context manager, you can now use the with statement. The running results are as follows:

__enter__ called
withCode block __exit__ called exc_type:None
exc_value:  None
exc_traceback:  NoneFile close operationwithEnd of the statementCopy the code


Comment out the division by 0 exception to see the result

__enter__ called
with__exit__ called exc_type: <class 'ZeroDivisionError'>
exc_value:  division by zero
exc_traceback:  <traceback object at 0x0000021F0780BCC8Traceback (most recent call last): File"C:\Users\Administrator\Desktop\pycode\withdemo.py", line 113.in <module>
    main()
  File "C:\Users\Administrator\Desktop\pycode\withdemo.py", line 106.in main
    a = 1 / 0
ZeroDivisionError: division by zero
Copy the code


This way, you don’t have to explicitly call the close method. The system calls it automatically, even if it encounters an exception.


Contextmanager based decorator

Python also provides a contextmanager decorator in the contextlib module, further simplifying the implementation of the contextmanager. The function is split in two by yield, with the statements before yield executed in the __enter__ method and the statements after yield executed in the __exit__ method. The value immediately following yield is the return value of the function.

# ContextManager decorator based
from contextlib import contextmanager

@contextmanager
def file_manager(name, mode) :
    print("file_manager() called")
    try:
        f = open(name, mode)
        yield f
    finally:
        f.close()
        print("File close operation")
Copy the code

call

with file_manager('test.txt'.'w') as f:
        print("With code block")
        f.write('hello world')
        # a = 1 / 0

print("End of statement with")
Copy the code

The results of

file_manager() called
withCode block file closure operationwithEnd of the statementCopy the code


conclusion

Python provides the with syntax to simplify subsequent cleanup of resource operations. It is an alternative to try/finally and is based on context managers. In addition, Python provides a contextmanager decorator to further simplify the implementation of the contextmanager. Class-based and ContextManager-based context managers, which are functionally identical. However, the class-based contextmanager is more flexible and suitable for large system development, whereas the contextmanager-based contextmanager is more convenient and concise and suitable for small and medium-sized applications.

Whichever you use, don’t forget the method__exit__()Or is itfinallyIt is especially important to free resources from blocks.


The source code

The source code has been uploaded to Gitee PythonKnowledge: PythonKnowledge Repository.

✍ code word is not easy, but also hope you heroes support ❤️.


The public,

Create a new folder X

Nature took tens of billions of years to create our real world, while programmers took hundreds of years to create a completely different virtual world. We knock out brick by brick with a keyboard and build everything with our brains. People see 1000 as authority. We defend 1024. We are not keyboard warriors, we are just extraordinary builders of ordinary world.