Small knowledge, big challenge! This article is participating in the creation activity of “Essential Tips for Programmers”.

This article also participated in the “Digitalstar Project” to win a creative gift package and creative incentive money

Thread alternatives

• subprocess
  • Skip threads altogether and use processes
  • Is the primary alternative to a derived process
  • Python2.4 introduced after
• multiprocessiong
  • Derived using the threading interface, using child processes
  • Allows multi-core or multi-CPU processes to be derived. The interface is very similar to threading
  • Python2.6 introduced after
• concurrent.futures
  • New asynchronous execution module
  • Task-level operations
  • Python3.2 introduced after

Multiple processes

• Interprocess communication (InterprocessCommunication, IPC)
• There is no shared state between processes
• Process creation
  • Generate Process instance objects directly, as in case 19

    import multiprocessing
    from time import sleep, ctime
    
    def clock(interval) :
        while True:
            print("The time is %s" % ctime())
            sleep(interval)
    
    if __name__ == '__main__':
        p = multiprocessing.Process(target=clock, args=(5, ))
        p.start()
    
        while True:
            print("sleeping......")
            sleep(1)
    Copy the code

    sleeping...... The time is Tue Aug 13 19:47:41 2019 sleeping...... sleeping...... sleeping...... sleeping...... sleeping...... The time is Tue Aug 13 19:47:46 2019 sleeping...... sleeping...... sleeping...... sleeping...... .Copy the code

  • Derived subclasses, case 20

    import multiprocessing
    from time import sleep, ctime
    
    class ClockProcess(multiprocessing.Process) :
        "Two functions are important: 1. Init constructor 2. Run"
        def __init__(self, interval) :
            super(ClockProcess, self).__init__()
            self.interval = interval
        def run(self) :
            while True:
                print("The time is %s" % ctime())
                sleep(self.interval)
    
    if __name__ == '__main__':
        p = ClockProcess(3)
        p.start()
    Copy the code

      The time is Tue Aug 13 19:48:49 2019
      The time is Tue Aug 13 19:48:52 2019
      The time is Tue Aug 13 19:48:55 2019
      The time is Tue Aug 13 19:48:58 2019
      The time is Tue Aug 13 19:49:01 2019
      The time is Tue Aug 13 19:49:04 2019
      ...
    Copy the code

• View pid, PPID, and their relationship in OS
  • 21 cases

    from multiprocessing import Process
    import os
    
    def info(title) :
        print(title)
        print("module name:", __name__)
        Get the id of the parent process
        print("parent process:", os.getppid())
        Get the id of the process itself
        print("process id:", os.getpid())
    
    def f(name) :
        info('function f')
        print('hello', name)
    
    if __name__ == '__main__':
        info('main line')
        p = Process(target=f, args=('bob', ))
        p.start()
        p.join()
    Copy the code

      main line
      module name: __main__
      parent process: 11900
      process id: 18832
      function f
      module name: __mp_main__
      parent process: 18832
      process id: 20868
      hello bob
    Copy the code

• Producer-consumer model
  • JoinableQueue
  • 22 cases

    import multiprocessing
    from time import ctime
    
    def consumer(input_q) :
        print("Into consumer:", ctime)
        while True:
            Item # processing
            item = input_q.get()
            print('pull', item, 'out of q')
            input_q.task_done()  Signal completion of task
        print("Out of consumer:", ctime()) Q.join () collects four task_done() signals, and the main process starts
    
    def producer(sequence, output_q) :
        print("Into procuder:", ctime())
        for item in sequence:
            output_q.put(item)
            print('put', item, 'into q')
        print('Out of procuder', ctime())
    
    # Set up process
    if __name__ == '__main__':
        q = multiprocessing.JoinableQueue()
        # Start the consumer process
        cons_p = multiprocessing.Process(target=consumer, args=(q, ))
        cons_p.daemon = True
        cons_p.start()
    
        Produce multiple items, with sequence representing the sequence of items to be sent to the consumer
        # In practice, this might be the output of a generator or produced by some other place
        sequence = [1.2.3.4]
        producer(sequence, q)
        Wait for all items to be processed
        q.join()
    Copy the code

      Into procuder: Tue Aug 13 19:50:38 2019
      put 1 into q
      put 2 into q
      put 3 into q
      put 4 into q
      Out of procuder Tue Aug 13 19:50:38 2019
      Into consumer: <built-in function ctime>
      pull 1 out of q
      pull 2 out of q
      pull 3 out of q
      pull 4 out of q
    Copy the code

  • Use of sentries in a queue, case 23

    import multiprocessing
    from time import ctime
    
    # set sentry problem
    def consumer(input_q) :
        print("Into consumer:", ctime())
        while True:
            item = input_q.get()
            if item is None:
                break
            print("pull", item, "out of q")
        print ("Out of consumer:", ctime()) ## This sentence is executed, and then the main process
    
    
    def producer(sequence, output_q) :
        print ("Into procuder:", ctime())
        for item in sequence:
            output_q.put(item)
            print ("put", item, "into q")
        print ("Out of procuder:", ctime())
    
    if __name__ == '__main__':
        q = multiprocessing.Queue()
        cons_p = multiprocessing.Process(target = consumer, args = (q,))
        cons_p.start()
    
        sequence = [1.2.3.4]
        producer(sequence, q)
    
        q.put(None)
        cons_p.join()
    Copy the code

      Into procuder: Tue Aug 13 19:51:23 2019
      put 1 into q
      put 2 into q
      put 3 into q
      put 4 into q
      Out of procuder: Tue Aug 13 19:51:23 2019
      Into consumer: Tue Aug 13 19:51:24 2019
      pull 1 out of q
      pull 2 out of q
      pull 3 out of q
      pull 4 out of q
      Out of consumer: Tue Aug 13 19:51:24 2019
    Copy the code

  • Improvement of sentry, case 24

    import multiprocessing
    from time import ctime
    
    def consumer(input_q) :
        print ("Into consumer:", ctime())
        while True:
            item = input_q.get()
            if item is None:
                break
            print("pull", item, "out of q")
        print ("Out of consumer:", ctime())
    
    def producer(sequence, output_q) :
        for item in sequence:
            print ("Into procuder:", ctime())
            output_q.put(item)
            print ("Out of procuder:", ctime())
    
    if __name__ == '__main__':
        q = multiprocessing.Queue()
        cons_p1 = multiprocessing.Process (target = consumer, args = (q,))
        cons_p1.start()
    
        cons_p2 = multiprocessing.Process (target = consumer, args = (q,))
        cons_p2.start()
    
        sequence = [1.2.3.4]
        producer(sequence, q)
    
        q.put(None)
        q.put(None)
    
        cons_p1.join()
        cons_p2.join()
    Copy the code

      Into procuder: Tue Aug 13 19:52:08 2019
      Out of procuder: Tue Aug 13 19:52:08 2019
      Into procuder: Tue Aug 13 19:52:08 2019
      Out of procuder: Tue Aug 13 19:52:08 2019
      Into procuder: Tue Aug 13 19:52:08 2019
      Out of procuder: Tue Aug 13 19:52:08 2019
      Into procuder: Tue Aug 13 19:52:08 2019
      Out of procuder: Tue Aug 13 19:52:08 2019
      Into consumer: Tue Aug 13 19:52:08 2019
      pull 1 out of q
      pull 2 out of q
      pull 3 out of q
      pull 4 out of q
      Out of consumer: Tue Aug 13 19:52:08 2019
      Into consumer: Tue Aug 13 19:52:08 2019
      Out of consumer: Tue Aug 13 19:52:08 2019
                  
                  
                  
    Copy the code

Finally, welcome to pay attention to my personal wechat public account “Little Ape Ruochen”, get more IT technology, dry goods knowledge, hot news