使用dask.array.core.map_blocks並行化3D numpy數組計算

Question

我有一個3D numpy數組（尺寸：深度，緯度，經度），我試圖使用沿每個緯度深度軸的數據進行一些並行計算 - 龍點，迄今爲止我一直不成功。我查看了dask.array.core.map_blocks的文檔，但沒有幫助。這是我在做什麼：

N2_dask = da.from_array(N2_naned, chunks=(49, 32, (12, 12, 12, 12, 12, 12))) 
zN2_dask = da.from_array(-zN2_agg[t], chunks=(49, 32, (12, 12, 12, 12, 12, 12))) 
lat_dask = da.from_array(lat_agg, chunks=(32)) 
lon_dask = da.from_array(lon_agg, chunks=((12, 12, 12, 12, 12, 12))) 

for j in range(len(lat_dask)): 
    for i in range(len(lon_dask)): 
     f = da.core.map_blocks(baroclinic.neutral_modes_from_N2_profile(
      zN2_dask[:, j, i], N2_dask[:, j, i], gsw.earth.f(lat_dask[j, i]), **kwargs)) 
     zphi, Rd, vd = f.compute() 

其中baroclinic.neutral_modes_from_N2_profile是我的功能。我得到一個錯誤如下：

AssertionErrorTraceback (most recent call last) 
<ipython-input-61-e58a7b54c470> in <module>() 
     6   print zN2_dask[:, j, i] 
     7   f = da.core.map_blocks(baroclinic.neutral_modes_from_N2_profile(
----> 8     zN2_dask[:, j, i], N2_dask[:, j, i], gsw.earth.f(lat_dask[j, i]), **kwargs)) 
     9   zphi, Rd, vd = f.compute() 

/home/takaya/.conda/envs/oceanmodes/lib/python2.7/site-packages/dask/array/core.pyc in __getitem__(self, index) 
    1023    return self 
    1024 
-> 1025   dsk, chunks = slice_array(out, self.name, self.chunks, index) 
    1026 
    1027   return Array(merge(self.dask, dsk), out, chunks, dtype=self._dtype) 

/home/takaya/.conda/envs/oceanmodes/lib/python2.7/site-packages/dask/array/slicing.pyc in slice_array(out_name, in_name, blockdims, index) 
    134 
    135  # Pass down to next function 
--> 136  dsk_out, bd_out = slice_with_newaxes(out_name, in_name, blockdims, index) 
    137 
    138  bd_out = tuple(map(tuple, bd_out)) 

/home/takaya/.conda/envs/oceanmodes/lib/python2.7/site-packages/dask/array/slicing.pyc in slice_with_newaxes(out_name, in_name, blockdims, index) 
    152 
    153  # Pass down and do work 
--> 154  dsk, blockdims2 = slice_wrap_lists(out_name, in_name, blockdims, index2) 
    155 
    156  # Insert ",0" into the key: ('x', 2, 3) -> ('x', 0, 2, 0, 3) 

/home/takaya/.conda/envs/oceanmodes/lib/python2.7/site-packages/dask/array/slicing.pyc in slice_wrap_lists(out_name, in_name, blockdims, index) 
    183  shape = tuple(map(sum, blockdims)) 
    184  assert all(isinstance(i, (slice, list, int, long)) for i in index) 
--> 185  assert len(blockdims) == len(index) 
    186  for bd, i in zip(blockdims, index): 
    187   check_index(i, sum(bd)) 

AssertionError: 

有誰能告訴我爲什麼這給了我一個AssertionError？先謝謝你！

Answer 1

結塊要麼是整數在整個陣列

chunks=(49, 32, 12) 

指定均勻chunkshape的元組或者，它應該是元組的元組，每個元組定義如何塊組成每個維度。

chunks=((20, 20, 9), (8, 8, 8, 8), (12, 24, 24, 12)) 

你的情況，你似乎是混合這些形式

chunks=(49, 32, (12, 12, 12, 12, 12, 12)) 

你可能打算執行下列操作之一：

chunks=(49, 32, 12) 
chunks=((49,), (32,), (12, 12, 12, 12, 12, 12)) 

鑑於最後一個維度被均勻分塊（全部12秒）我推薦使用(49, 32, 12)