鎖定進程，直到子進程到達特定點

Question

我正在並行運行多個子進程，但我需要鎖定每個進程，直到子進程給出輸出（通過打印函數）。子進程正在運行已打包到可執行文件的python腳本。

的代碼看起來是這樣的：

import multiprocessing as mp 
import subprocess 
import os 


def main(args): 

    l,inpath = args 
    l.acquire() 
    print "Running KNN.exe for files in %s" % os.path.normpath(inpath).split('\\')[-1] 

    #Run KNN executable as a subprocess 
    subprocess.call(os.path.join(os.getcwd(), "KNN.exe")) 
    #This is where I want to wait for any output from the subprocess before releasing the lock 
    l.release() 

    #Here I would like to wait until subprocess is done then print that it is done 
    l.acquire() 
    print "Done %s" % os.path.normpath(inpath).split('\\')[-1] 
    l.release() 


if __name__ == "__main__": 
    #Set working directory path containing input text file 
    os.chdir("C:\Users\Patrick\Google Drive\KNN") 
    #Get folder names in directory containing GCM input 
    manager = mp.Manager() 
    l = manager.Lock() 
    gcm_dir = "F:\FIDS_GCM_Data_CMIP5\UTRB\UTRB KNN-CAD\Input" 
    paths = [(l, os.path.join(gcm_dir, folder)) for folder in os.listdir(gcm_dir)]  
    #Set up multiprocessing pool 
    p = mp.Pool(mp.cpu_count()) 
    #Map function through input paths 
    p.map(main, paths) 

因此我們的目標是鎖定的過程，這樣的子進程可以直到接收輸出運行。之後鎖可以被釋放，子進程可以繼續，直到完成，然後我想打印它是完整的。

我的問題是，如何才能在釋放進程上的鎖（等於多個）之前等待子進程中的單個（且唯一）輸出？

此外，如何等待進程終止，然後打印它是完整的？

Answer 1

您的代碼使用call方法，該方法已經等待子流程完成（這意味着所有輸出已經生成）。我從你的問題中推斷出你希望能夠區分輸出第一次寫入和子進程完成時的區別。下面是你的代碼我建議修改在線：

def main(args): 

    l,inpath = args 
    l.acquire() 
    print "Running KNN.exe for files in %s" % os.path.normpath(inpath).split('\\')[-1] 

    #Run KNN executable as a subprocess 
    #Use the Popen constructor 
    proc = subprocess.Popen(os.path.join(os.getcwd(), "KNN.exe"), stdout=subprocess.PIPE) 

    #This is where I want to wait for any output from the subprocess before releasing the lock 
    # Wait until the subprocess has written at least 1 byte to STDOUT (modify if you need different logic) 
    proc.stdout.read(1) 

    l.release() 

    #Here I would like to wait until subprocess is done then print that it is done 
    #proc.wait() 
    (proc_output, proc_error) = proc.communicate() 

    l.acquire() 
    print "Done %s" % os.path.normpath(inpath).split('\\')[-1] 
    l.release() 

注意上面不承擔你想要做的比檢查它已經產生的其他子進程的任何輸出。如果你想對輸出做的任何事情都不如上面那麼簡單（消耗1個字節然後放在地板上），那麼proc.stdout（這是一個file對象）應該代表子進程在運行時產生的所有東西。