Purpose: Store the child process return value
Note: If you just store the return value of the child process, you can use Pool’s apply_async asynchronous process pool; of course, you can also use Process, the usage is the same as in threading, only the former is introduced here.
Example: When all child processes in the process pool have finished executing, output the return value of each child process.
Wrong way of writing:
Description: This will cause blocking, because the get() method can only be called after the child process has finished running, otherwise it will block waiting. If written in the for loop content, the equivalent of becoming a synchronization, the execution efficiency will be very low.
Purpose: The parent process gets the child process return value in real time, ending all processes as a token.
During the execution of the child process, the return value is continuously obtained and verified. If the return value is True, all processes are returned.
Description: A total of 50,000 child processes are executed (the number of concurrent is 10), and when one of the child processes returns True, the process pool ends. Because apply_async is used as an asynchronous process, after executing the add subprocess operation of the for loop (just adding and not executing all the child processes), you can directly execute the while code to determine in real time whether the return value of the child process is True or not. The words end all processes.
Advantages: You don’t have to wait until all child processes have finished and then end the program. As long as you get the desired result, you can end early and save resources.
Insufficient: When the child process that needs to be executed is very large, it does not apply, because the for loop takes a long time to add the child process. Although it is asynchronous, it also needs to wait for the for loop to add the child process to end the execution of the while code. , so it will be slower.
Multi-threaded + multi-process, in the process of adding the execution of the child process, constantly get the return value and check, if the return value is True, then all the processes.
Execution process: using multi-threading, create a thread that executes the pool_th function, one executes the function of the result_th function, the pool_th function is used to add the process pool, the process execution function is started, and the child process object is stored in the queue, and the result_th() function is used instead. Stop the child process object from the queue, call the get () method to get the return value. When it finds that the return value of the child process is True, it ends all processes and ends the thread.
Advantages: Make up for the shortcomings of the example (a), even if the number of child processes in the for loop is large, it can improve performance, because the for loop and the judgment of the child process return value simultaneously.
[Multiprocessing Series] Shared Resources
[[Multiprocessing Series] Subprocess Return Value] (http://thief.one/2016/11/24/Multiprocessing%E5%AD%90%E8%BF%9B%E7%A8%8B%E8%BF%94 %E5%9B%9E%E5%80%BC/)
[[Multiprocessing Series] Multiprocessing Basics] (http://thief.one/2016/11/23/Python-multiprocessing/)