如何查看已完成或剩餘的map_async作業數量？

Question

我使用iPython的並行處理工具進行大型地圖操作。在等待地圖操作完成時，我想向用戶顯示已完成的作業數量，正在運行的作業數量以及剩餘的作業數量。我怎樣才能找到這些信息？

這就是我所做的。我創建一個使用本地引擎的配置文件，並啓動兩名工作人員。在外殼：

$ ipython profile create --parallel --profile=local 
$ ipcluster start --n=2 --profile=local 

下面是客戶端的Python腳本：從腳本輸出

#!/usr/bin/env python 

def meat(i): 
    import numpy as np 
    import time 
    import sys 
    seconds = np.random.randint(2, 15) 
    time.sleep(seconds) 
    return seconds 

import time 
from IPython.parallel import Client 

c = Client(profile='local') 
dview = c[:] 

ar = dview.map_async(meat, range(4)) 
elapsed = 0 
while True: 
    print 'After %d s: %d running' % (elapsed, len(c.outstanding)) 
    if ar.ready(): 
     break 
    time.sleep(1) 
    elapsed += 1 
print ar.get() 

例子：

After 0 s: 2 running 
After 1 s: 2 running 
After 2 s: 2 running 
After 3 s: 2 running 
After 4 s: 2 running 
After 5 s: 2 running 
After 6 s: 2 running 
After 7 s: 2 running 
After 8 s: 2 running 
After 9 s: 2 running 
After 10 s: 2 running 
After 11 s: 2 running 
After 12 s: 2 running 
After 13 s: 2 running 
After 14 s: 1 running 
After 15 s: 1 running 
After 16 s: 1 running 
After 17 s: 1 running 
After 18 s: 1 running 
After 19 s: 1 running 
After 20 s: 1 running 
After 21 s: 1 running 
After 22 s: 1 running 
After 23 s: 1 running 
[9, 14, 10, 3] 

正如你看到的，我可以得到的目前數量正在運行的作業，但不包括已完成（或剩餘）的作業數量。我怎麼知道有多少map_async的工作已經完成？

Answer 1

AsyncResult具有msg_ids屬性。突出的工作是與rc.outstanding交集，和完成作業都有所差異：

msgset = set(ar.msg_ids) 
completed = msgset.difference(rc.outstanding) 
pending = msgset.intersection(rc.outstanding)