連接到dask.distributed羣集時發生Pickle錯誤

Question

這是我簡單的代碼。試圖運行我的第一個程序。

from dask.distributed import Client 
client = Client('192.168.1.102:8786') 


def inc(x): 
    return x + 1 

x = client.submit(inc, 10) 
print(x.result()) 

嘗試使用這個命令來運行這段代碼時：

$python3 filename.py 

我geing此錯誤：

/usr/local/lib/python3.4/dist-packages/distributed/protocol/pickle.py 
- INFO - Failed to serialize <function inc at 0x7f678ad05840> Traceback (most recent call last): File 
"/usr/local/lib/python3.4/dist-packages/distributed/protocol/pickle.py", 
line 33, in dumps 
    return cloudpickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL) AttributeError: 'module' object has no attribute 'dumps' 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): File 
"/usr/local/lib/python3.4/dist-packages/distributed/protocol/pickle.py", 
line 43, in dumps 
    return cloudpickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL) AttributeError: 'module' object has no attribute 'dumps' Traceback 
(most recent call last): File 
"/usr/local/lib/python3.4/dist-packages/distributed/protocol/pickle.py", 
line 33, in dumps 
    return cloudpickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL) AttributeError: 'module' object has no attribute 'dumps' 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): File "dis.py", line 8, in 
<module> 
    x = client.submit(inc, 10) File "/usr/local/lib/python3.4/dist-packages/distributed/client.py", line 
643, in submit 
    loose_restrictions, priority={skey: 0}) File "/usr/local/lib/python3.4/dist-packages/distributed/client.py", line 
1235, in _graph_to_futures 
    'tasks': valmap(dumps_task, dsk3), File "/usr/local/lib/python3.4/dist-packages/toolz/dicttoolz.py", line 84, 
in valmap 
    rv.update(zip(iterkeys(d), map(func, itervalues(d)))) File "/usr/local/lib/python3.4/dist-packages/distributed/worker.py", line 
812, in dumps_task 
    return {'function': dumps_function(task[0]), File "/usr/local/lib/python3.4/dist-packages/distributed/worker.py", line 
779, in dumps_function 
    b = dumps(func) File "/usr/local/lib/python3.4/dist-packages/distributed/protocol/pickle.py", 
line 43, in dumps 
    return cloudpickle.dumps(x, protocol=pickle.HIGHEST_PROTOCOL) AttributeError: 'module' object has no attribute 'dumps' 

Answer 1

通常酸洗錯誤跡象表明您DASK的一些元素。分佈式網絡（worker，scheduler或client）的Python版本不匹配。也許你的工作人員沒有意識到它在Python 2下運行？

該錯誤特別說明cloudpickle庫沒有dumps方法，這很奇怪。據我所知，cloudpickle一直有dumps函數。你的環境中是否有一個奇怪的cloudpickle庫，或者是一個非常舊的版本？

如果你只是想的東西出來，你也可以通過omiting調度

from dask.distributed import Client 

# client = Client('scheduler-address:8786') 
client = Client() # create local "cluster"