Today we are going to start a new series, which is python multithreading and multi-process implementation. This part may be some beginners are still fuzzy, all know that python multithreading is false, but do not know what is going on, first let’s look at an example to look at python multithreading implementation.
import threading
import time
def say(name):
print('Hello %s at %s' %(name,time.ctime()))
time.sleep(2)
print(End %s at %s %(name,time.ctime()))
def listen(name):
print('Hello %s at %s' % (name,time.ctime()))
time.sleep(4)
print(End %s at %s % (name,time.ctime()))
if __name__ == '__main__':
t1 = threading.Thread(target=say,args=('tony')),Thread is a class that instantiates t1 objects
t1.start() # thread execution
t2 = threading.Thread(target=listen, args=('simon')),Create a thread object t2
t2.start()
print("End of program ====================="Results: Hello Tony at Thu Apr 25 16:46:22 2019 Hello Simon at Thu Apr 25 16:46:22 2019 ===================== End Tony at Thu Apr 25 16:46:22 2019 Thu Apr 25 16:46:26 2019 Process Finished withexit code 0
Copy the codePython multithreading is implemented through the Thread class of the Threading module. Thread(target=say,args=(‘ Tony ‘,)) #Thread is a class that instantiates a t1 object
Let's analyze the result of the above code: Hello Tony at Thu Apr 25 16:46:22 2019 -- T1 thread Execution Hello Simon at Thu Apr 25 16:46:22 2019 -- T2 thread execution program end ===================== Tony at Thu Apr 25 16:46:26 2019 --sleep Tony at Thu Apr 25 16:46:26 2019 --sleep The T2 thread executes Process Finished withexitCode 0 -- end of main threadCopy the codeWe can see that the main thread print does not wait for both t1 and T2 threads to complete, because the main thread and T1 and T2 threads run at the same time. However, the main process does not exit until the non-daemon child thread terminates.
This is actually the simplest use of multiple threads in Python, but some people may have the same requirement as me. Usually in development, we need print at the end of the main thread to indicate that all processes are finished. This introduces the concept of a join.
import threading
import time
def say(name):
print('Hello %s at %s' %(name,time.ctime()))
time.sleep(2)
print(End %s at %s %(name,time.ctime()))
def listen(name):
print('Hello %s at %s' % (name,time.ctime()))
time.sleep(4)
print(End %s at %s % (name,time.ctime()))
if __name__ == '__main__':
t1 = threading.Thread(target=say,args=('tony')),Thread is a class that instantiates t1 objects
t1.start() # thread execution
t2 = threading.Thread(target=listen, args=('simon')),Create a thread object t2
t2.start()
t1.join() #join, etc. t1 child thread terminates, main thread prints and terminates
t2.join() #join t2 child thread terminates, main thread prints and terminates
print("End of program ====================="Results: Tony at Thu Apr 25 16:57:32 2019 Hello Simon at Thu Apr 25 16:57:32 2019 Thu Apr 25 16:57:36 2019 program end = = = = = = = = = = = = = = = = = = = = =Copy the codeIf the child thread does not join, the main thread will print as well, but the main thread will not finish. The main thread still needs to finish after the non-daemon child thread has finished.
In both cases, the main process has to wait for the non-daemon child thread to terminate before the main thread terminates. Did we notice that I said “non-daemon child thread”, so what is a non-daemon child thread? The default child thread is the non-daemon child thread of the main thread, but sometimes we need to follow the main thread to exit when the main process terminates, regardless of whether the thread terminates or not, so we introduce the concept of “daemon thread”.
If a child thread is set up as a daemon thread, the main thread actually doesn’t care about that child thread, and when all the other non-daemon threads end, the main thread will exit, and the daemon thread will exit with the main thread, guarding the main thread, that’s what daemons mean
Let’s take a look at the code. We’ll discuss daemons in two different ways to give you a better understanding of them. One more thing: this method must be set before the start method
1. Set the T1 thread as the daemon thread and view the result
import threading
import time
def say(name):
print('Hello %s at %s' %(name,time.ctime()))
time.sleep(2)
print(End %s at %s %(name,time.ctime()))
def listen(name):
print('Hello %s at %s' % (name,time.ctime()))
time.sleep(4)
print(End %s at %s % (name,time.ctime()))
if __name__ == '__main__':
t1 = threading.Thread(target=say,args=('tony')),Thread is a class that instantiates t1 objects
t1.setDaemon(True)
t1.start() # thread execution
t2 = threading.Thread(target=listen, args=('simon')),Create a thread object t2
t2.start()
print("End of program ====================="Results: Hello Tony at Thu Apr 25 17:11:41 2019 Hello Simon at Thu Apr 25 17:11:41 2019 ===================== End Tony at Thu Apr 25 17:11:41 2019 Simon at Thu Apr 25 17:11:45 2019Copy the codeNote the order of execution. If you set T1 to Daemon, the main thread does not care about the state of T1 and waits for t2 to end, and then the main thread ends because t2 is 4 seconds, T1 is 2 seconds, and while waiting for T2 to end, t1 terminates itself, so the result is: Tony at Thu Apr 25 14:11:54 2019 Hello Simon at Thu Apr 25 14:11:54 2019 =============== End Tony at Thu Apr 25 14:11:56 2019 Thu Apr 25 14:11:58 2019 Simon at Thu Apr 25 14:11:58 2019Copy the code2. Set T2 as the daemon thread
import threading
import time
def say(name):
print('Hello %s at %s' %(name,time.ctime()))
time.sleep(2)
print(End %s at %s %(name,time.ctime()))
def listen(name):
print('Hello %s at %s' % (name,time.ctime()))
time.sleep(4)
print(End %s at %s % (name,time.ctime()))
if __name__ == '__main__':
t1 = threading.Thread(target=say,args=('tony')),Thread is a class that instantiates t1 objects
t1.start() # thread execution
t2 = threading.Thread(target=listen, args=('simon')),Create a thread object t2
t2.setDaemon(True)
t2.start()
print("End of program ====================="Results: Hello Tony at Thu Apr 25 17:15:36 2019 Hello Simon at Thu Apr 25 17:15:36 2019 ===================== End Tony at Thu Apr 25 17:15:36 2019 17:15:38 2019Copy the codeNote the order of execution: if t2 is set to Daemon, the main thread does not care about the running status of T2 and waits for t1 to end. When T1 ends, the main thread ends. Because t2 is 4 seconds and T1 is 2 seconds, the main thread cannot end itself while waiting for T1 to end. Hello Tony at Thu Apr 25 14:14:23 2019 Hello Simon at Thu Apr 25 14:14:23 2019 == == == == == == == == == == = End Tony at Thu Apr 25 14:11:58 2019 will not print because the main thread is waiting for t1 to finish, t2 cannot finish by itself, t2 is 4 seconds, T1 is 2 secondsCopy the codeI don’t know if you’re clear about the python implementation of multithreading and the use of join daemon threads.
Main methods:
Join () : The parent thread of the child thread is blocked until the child thread finishes running.setDaemon(True) : Declares threads as daemons, which must be set before the start() method is called. If not, applications will be suspended indefinitely. This method is basically the opposite of join. When we run a program, we execute a main thread, and if the main thread creates a child thread, the main thread and the child thread run separately, and then when the main thread finishes and wants to exit, it checks whether the child thread is complete. If the fruit thread is incomplete, the main thread will wait for the child thread to complete before exiting. But sometimes we need to exit with the main thread as soon as the main thread is finished, regardless of whether the thread is finished or not, and then we can use itsetThe Daemon method!Copy the codeOther methods:
Run (): automatically executes the run method of the thread object after the thread is scheduled by the CPU. Start (): starts the thread activity. IsAlive (): Returns whether the thread is active. GetName (): Returns the thread name.setName(): Sets the thread Name. Some methods provided by the threading module: threading.currentthread (): returns the currentThread variable. Threading.enumerate (): Returns a list of running threads. Running refers to the thread after starting and before ending, excluding the thread before starting and after ending. Threading.activecount (): Returns the number of threads running as len(threading.enumerate()).Copy the codeIn the example above, we noticed that if two tasks if order to 6 s end, if it is a multithreaded execution 4 s end, the performance is improved, but we want to know the performance improvement is, in fact, due to the CPU concurrent implementation performance, namely the CPU thread switches (multichannel technology), and is not really much CPU parallel execution.
I mentioned parallelism and concurrency, what’s the difference between the two? Concurrency: refers to the ability of a system to handle multiple tasks (CPU switching, multi-channel technology) parallelism: refers to the ability of a system to handle multiple tasks at the same time (CPU processing multiple tasks at the same time) Parallelism is a subset of concurrency
So why can’t Python do true parallelism in multithreading? The GIL is a thread that runs in parallel on multiple cpus. The GIL is a thread that runs in parallel on multiple cpus.
GIL: Global interpreter lock No matter how many threads you open and how many cpus you have, Python will only allow one thread (competing threads) to have the GIL running on one CPU at a time. When the thread encounters an I/O wait or arrives polling time, the CPU will switch. CPU switching consumes time and resources, so computation-intensive functions (such as addition, subtraction, multiplication, and division) are not suitable for multithreading because there are too many CPU threads switching, and IO intensive functions are suitable for multithreading.
Tasks: IO intensive (each thread will have to wait, and if there is a wait, thread switching is suitable), you can also use multiple processes + coroutine computation-intensive (thread is not waiting, and then switch, is useless switch), Python is not suitable for this kind of functionality
In the previous example, we simulated the sleep operation, which is actually equivalent to the IO. In this scenario, multi-threading can increase performance, but if we use multi-threading to calculate data calculation, the performance will be reduced.
Here’s an original explanation from GIL:
In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly CPython's memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.)Copy the codeMy personal opinion is appreciated