多處理簡單功能不起作用，但爲什麼

Question

我想多處理系統命令，但不能讓它與一個簡單的程序一起工作。該功能runit（CMD）工作正常，但...

#!/usr/bin/python3 
from subprocess import call, run, PIPE,Popen 
from multiprocessing import Pool 
import os 
pool = Pool() 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

#print(runit('ls -l')) 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

results = pool.map(runit, it) 

它輸出：

Process ForkPoolWorker-1: 
Process ForkPoolWorker-2: 
Traceback (most recent call last): 
Traceback (most recent call last): 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap 
    self.run() 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run 
    self._target(*self._args, **self._kwargs) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 108, in worker 
    task = get() 
    File "/usr/lib/python3.5/multiprocessing/queues.py", line 345, in get 
    return ForkingPickler.loads(res) 
AttributeError: Can't get attribute 'runit' on <module '__main__' from './syscall.py'> 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap 
    self.run() 
    File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run 
    self._target(*self._args, **self._kwargs) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 108, in worker 
    task = get() 
    File "/usr/lib/python3.5/multiprocessing/queues.py", line 345, in get 
    return ForkingPickler.loads(res) 
AttributeError: Can't get attribute 'runit' on <module '__main__' from './syscall.py'> 

然後以某種方式等待和什麼也不做，當我按下Ctrl + C幾次它吐出來：

^CProcess ForkPoolWorker-4: 
Process ForkPoolWorker-6: 
Traceback (most recent call last): 
    File "./syscall.py", line 17, in <module> 
Process ForkPoolWorker-5: 
    results = pool.map(runit, it) 
    File "/usr/lib/python3.5/multiprocessing/pool.py", line 260, in map 
... 
    buf = self._recv(4) 
    File "/usr/lib/python3.5/multiprocessing/connection.py", line 379, in _recv 
    chunk = read(handle, remaining) 
KeyboardInterrupt 

Answer 1

我不知道，因爲我知道這個問題是Windows相關的（和我沒有訪問Linux中reprocude），但爲了可以移植，你必須換你多依賴於的命令或者它的方式蟒蛇衝突滋生的過程：即固定例如運行良好的窗口（並應工作確定在其他平臺上也一樣）：

from multiprocessing import Pool 
import os 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

#print(runit('ls -l')) 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

if __name__=="__main__": 
    # all calls to multiprocessing module are "protected" by this directive 
    pool = Pool() 

（更緊密地學習中的錯誤消息，現在我敢肯定，只是移動pool = Pool()後的runit的聲明將修復它，以及在Linux上，但包裹在__main__修復+使得便攜式）

不過，請注意，您的多隻創建一個新的進程，所以你用線程池（Threading pool similar to the multiprocessing Pool?）更好：創建進程的線程SES，像這樣：

from multiprocessing.pool import ThreadPool # uses threads, not processes 
import os 

def runit(cmd): 
    proc = Popen(cmd, shell=True,stdout=PIPE, stderr=PIPE, universal_newlines=True) 
    return proc.stdout.read() 

it = [] 
for i in range(1,3): 
    it.append('ls -l') 

if __name__=="__main__": 
    pool = ThreadPool() # ThreadPool instead of Pool 
    results = pool.map(runit, it) 
    print(results) 
     results = pool.map(runit, it) 
     print(results) 

後者溶液是更輕巧，並且不太問題傾向（多是一個微妙的模塊來處理）。您將能夠使用對象，共享數據等......而無需Manager對象，其他優勢