preface

Today's computers, whether single CPU or multiple CPU, are basically multi-core, using multi-process, you can run full multi-core CPU. Python provides a very useful multiprocess module called Multiprocessing. You only need to define a function and Python will do everything else for you. With this package, you can easily convert from single process to concurrent execution.Copy the code

1/ Single process

   If we create a small number of processes, we can do the following:
   import multiprocessing
   import time

   def func(msg) : # this function will be executed three times, each time will print the passed parameter MSG, each time after printing, regret 1 second sleep
      for i in range(3) :print( msg )
          time.sleep(1)

   if __name__ == "__main__":
      Create a process that includes function functions and parameters
      p = multiprocessing.Process(target=func,args=("hello", )) 
      p.start()  # Execute the process
      p.join()   The parent process waits for all its children to finish executing.
      print( "Sub-process done.")   The child process is complete
Copy the code

2/ Use process pools

   # Yes, you read that right, not thread pool, but process pool.
   # It allows you to run full CPU cores and is very simple to use.
   Use apply_async. If async is left behind, it will block.

   # processes=4 specifies the maximum number of concurrent processes
   # When setting this parameter, we need to know in advance how many cores our computer has

   import multiprocessing
   import time

   def f1(msg) :
       for i in xrange(3) :print( msg )
           time.sleep(1)

   if __name__ == "__main__":
       # If the computer has 4 cores
       pool = multiprocessing.Pool(processes=4)  # Multiprocessing.pool (4)
       for i in range(10):
           msg = "hello %d" %(i)
           pool.apply_async(f1, (msg, ))
       pool.close()  # stop accepting new child processes from the pool
       pool.join()  # daemon
       print( "Sub-process(es) done." ) The child process is complete
Copy the code

3/ Use Pool and pay attention to the result

    # More often, not only do we need multi-process execution, but we also need to pay attention to the results of each process,
    # code is as follows:

    import multiprocessing
    import time

    def func(msg) :
        for i in range(3) :print( msg )
            time.sleep(1)
        return "done " + msg

    if __name__ == "__main__":
        pool = multiprocessing.Pool(processes=4)  # Multiprocessing.pool (4)
        result = []
        for i in range(10):
            msg = "hello %d" %(i)
            result.append(pool.apply_async(func, (msg, )))
        pool.close()
        pool.join()
        for res in result:
            print( res.get() )
        print( "Sub-process(es) done." ) The child process is complete
Copy the code