Multi-process programming and interprocess communication Advantages and disadvantages of multi-process Process three-state five-state process priority Process Characteristics Orphan process Zombie process Requirements Understanding multi-process programming process-related functions multi-process module Create custom inherited Process class Process Pool technology Pool function Inter-process communication (IPC) Pipeline communication Multi-process pipeline transmission data example Message queue Single-process example multi-process message queue transmission data shared memory signal communication

Multi – process programming and interprocess communication

• Meaning: Make full use of computer resources to improve the speed of the program
• Definition: An application that increases the speed of a computer by using multiple computer cores to perform multiple tasks simultaneously
• Implementation plan: multi-process multi-thread
• Parallelism: The computer processes multiple tasks at the same time
• Concurrency: Processing multiple tasks at the same time, with the kernel switching between tasks as if they were all still running, but in reality the kernel can only handle one of them at a time

Advantages and disadvantages of multiple processes

Process

Procedure: ps-aux Procedure tree: pstree Procedure Parent process: ps-ajx procedure

Three states

Five states (new state and termination state are added on the basis of three states)

• New state: The process of creating a new process to obtain resources
• Terminating state: The process in which a process finishes execution and resources are released and reclaimed

Process priority

Process characteristics

1. Processes run independently without affecting each other
2. Processes are the smallest unit of operating system resource allocation
3. Each process occupies a certain amount of virtual memory

An orphan

zombies

1. The child process exits before the parent process, but the parent process does not handle the exit status of the child process, so the child process becomes a zombie process
2. Zombie processes store a small amount of PCB information in the memory. A large number of zombie processes consume system resources. Therefore, avoid zombie processes

• How can zombie processes be avoided

  • Handles the exit status of the child process

    pid, status = os.wait()Copy the code

     

    Block in parent process and wait for child process to exit

    The return value:

To understand

Multiprocess programming

4 # Parameter: None 5 # Return value: 7. Import OS 9 PID = os.fork() 10 PID = os.fork() 11 Print (pid) 9 print(pid) 10Copy the code

• The child copies all of the parent’s code, including the memory generated before the fork
• The child executes from the next sentence of fork, independently of the parent
• The execution order of the parent and child processes is not necessarily the same
• Parent and child processes often choose to execute different code depending on the return value of fork. So the if structure is almost standard for forks
• The space of the parent and child processes is independent. Operations are the contents of the space and do not affect each other
• Child processes also have their own characteristics, such as PID number, PCB, command set, etc.

Process-related functions

Import OS pid = os.fork() if pid < 0: print(" create process failed ") elif pid == 0: Print (" my true PID is: ",os.getpid()," my parent PID is: ", os.getppId ()) else: Print (" I am the parent of the code, the current variable pid is: ",pid," my real pid is: ",os.getpid()) "'''''' I am the parent of the code, the current variable pid is: 10992 my real pid is: My parent PID is 10991 '''''' "" # If PID 10992 is different from its parent PID, the child becomes an orphanCopy the code

Multiprocess module

import multiprocessing

from multiprocessing import Process

Process()

Def worker(SEC,name): for I in range(3): def worker(SEC,name): for I in range(3): sleep(sec) print("I'm %s"%name) print("I'm working..." ) p = Process(target = worker,args = (2,),\ kwargs = {'name':'Daivl'},name = "Worker") p.start() print("Process Print ("Process is alive:", p.iss_alive ()) p.in (3) print("Process is alive:", p.id) print("==================")Copy the code

Create a custom inherited Process class

1. Inheriting the Process

2. Write your own__init__, while loading the parent class__init__methods

3. rewriterunMethod, which can be run automatically by calling start on the generated object

from import Process import time class ClockProcess(Process): def __init__(self,value): self.value = value super(ClockProcess,self).__init__() def run(self): for i in range(5): Print (" current time is ",time.ctime()) time.sleep(self.value) # run p.start() p.in ()Copy the code

Process pool technology

The Pool function

• Pool(Processes)

  • Function: Create a process pool object
  • Indicates how many processes are in the process pool

• pool.apply_async(func, args, kwds)

  • Run the following command to add time to a process pool queue

  • parameter

    • Func: event function
    • Args: passes arguments to func as tuples
    • KWDS: pass parameters to func in dictionary form

  • Returns the value of an object

• pool.apply(func, args, kwds)

  • Run the following command to add time to a process pool queue

  • parameter

    • Func: event function
    • Args: passes arguments to func as tuples
    • KWDS: pass parameters to func in dictionary form

• pool.close()

  • Run the following command to disable a process pool

• pool.join()

  • Run the following command to reclaim a process pool

• pool.map(func,iter)

  • Function: The event to be done is put into the process pool

  • parameter

    • Func: Function to be executed
    • Iter: iterates

  • Return value: Result column of multiple returns

from multiprocessing import Process import time class ClockProcess(Process): def __init__(self,value): self.value = value super(ClockProcess,self).__init__() def run(self): for i in range(5): Print (" current time is ",time.ctime()) time.sleep(self.value) # run p.start() p.in ()Copy the code

Interprocess communication (IPC)

Pipeline communications

Communication principle: open up pipeline space in memory, generate pipeline operation object, multiple processes use the “same” pipeline object for operation, can realize communication

multiprocessing –> Pipe

• fd1,fd2 = Pipe(duplex = True)

  • Create a pipe

  • parameter

    • The default is a bidirectional pipe
    • Set False to one-way pipe

  • The return value

    • If the pipe is two-way, both can read and write
    • If the pipe is one-way, fd1 is read only and fd2 is write only

• fd.recv()

  • Function: Reads information from a pipe
  • Return value: What was read
  • If the pipe is empty, it is blocked

• fd.send(data)

  • Function: Write content to a pipe
  • Parameter: What to write
  • You can send any Python data type

Example of multi-process piped data transfer

From multiprocessing import Process,Pipe import time, OS # create Pipe object fd1, fd2 = Pipe() def fun(name): Time.sleep (1) fd2.send(os.getppId ()) jobs = [] # create 5 child processes for I in range(5): p = Process(target = fun,args = (i,)) jobs.append(p) p.start() for i in range(5): data = fd1.recv() print(data) for i in jobs: i.join()Copy the code

The message queue

Queue: First in, first out, in order

Communication principle: build queue data structure model in memory. Multiple processes can queue content in and out in the same order as they queue content in

• Create a queue

  q = Queue(maxsize = 0)

  • Function: Create a queue message
  • Parameter: Indicates the maximum number of messages to store. By default, storage is allocated according to memory
  • Return value: queue object

• q.put(data, [block, timeout])

  • Function: Stores messages like a queue

  • parameter

    • Data: the contents to be saved
    • Block: The default queue is blocked when it is full. If the value is set to False, the queue is not blocked
    • Timeout: indicates the timeout period

• data = q.get([block, timeout])

  • Function: Get queue messages

  • parameter

    • Block: By default, the queue is blocked when empty, but not blocked when False
    • Timeout: indicates the timeout period

  • Return value: Returns the retrieved content

• Q.null () : checks whether the queue is full

• Q.mpty () : checks whether the queue is empty

• Q.size () : Determines the number of messages in the queue

• Q.close: closes the queue

Single process example

#！《 单进程 》
from multiprocessing import Queue
from time import sleep
​
# 创建队列，可以放3条消息
q = Queue(3)
# 存一条消息
q.put(1)
sleep(0.5)
# 判断队列是否为空
print(q.empty())
q.put(2)
# 判断队列是否满
print(q.full())
q.put(3)
# 输出队列消息数量
print(q.qsize())
# 输出
print(q.get())
q.close()Copy the code

Multi-process message queues transfer data

Def fun1() from multiprocessing import Queue,Process from time import sleep sleep(1) q.put({"a":1,"b":2}) def fun2(): Print () p1 = Process(target = fun1) p2 = Process(target = fun2) p1.start() p2.start() p2.start() p1.join() p2.join()Copy the code

The Shared memory

Signal communication

One process sends a signal to another to transmit some information, and the receiver acts accordingly

$kill -l View system signal description

$kill -9 PID sends signals to the process

In Pythonimport signalYou can pick up the signal

• os.kill(pid, sig)

  • Function: send signal

  • parameter

    • Pid: INDICATES the PID number to send the signal
    • Sig: indicates the signal name

Import OS import signal os.kill(12345, signal.sigkill) #Copy the code

Banl



www.cnblogs.com/BanL/