並行與scikit學習pairwise_distances n_jobs

Question

感謝菲利普雲的偉大answer to a previous question，我去了，挖成的源代碼在scikit pairwise_distances。

相關部分是：

def pairwise_distances(X, Y=None, metric="euclidean", n_jobs=1, **kwds): 
    if metric == "precomputed": 
     return X 
    elif metric in PAIRWISE_DISTANCE_FUNCTIONS: 
     func = PAIRWISE_DISTANCE_FUNCTIONS[metric] 
     if n_jobs == 1: 
      return func(X, Y, **kwds) 
     else: 
      return _parallel_pairwise(X, Y, func, n_jobs, **kwds) 
    elif callable(metric): 
     # Check matrices first (this is usually done by the metric). 
     X, Y = check_pairwise_arrays(X, Y) 
     n_x, n_y = X.shape[0], Y.shape[0] 
     # Calculate distance for each element in X and Y. 
     # FIXME: can use n_jobs here too 
     D = np.zeros((n_x, n_y), dtype='float') 
     for i in range(n_x): 
      start = 0 
      if X is Y: 
       start = i 
      for j in range(start, n_y): 
       # distance assumed to be symmetric. 
       D[i][j] = metric(X[i], Y[j], **kwds) 
       if X is Y: 
        D[j][i] = D[i][j] 
     return D 

它是正確的，從這個明白，如果我要計算成對距離矩陣，如：

matrix = pairwise_distances(foo, metric=lambda u,v: haversine(u,v), n_jobs= -1)

其中haversine(u,v)是一個函數，計算兩個點之間的距離半正矢和該功能是在不PAIRWISE_DISTANCE_FUNCTIONS，該計算將不是即使n_jobs= -1被並行化？

我意識到#FIXME評論似乎暗示這一點，但我要確保我沒有瘋，因爲它似乎有些奇怪會有說明該計算實際上不會拋出並行無警示信息當您通過n_jobs= -1的可調用函數不在PAIRWISE_DISTANCE_FUNCTIONS中時。

Answer 1

證實。與n_jobs= -1一起傳遞一個可調用的metric不會導致並行如果調用不PAIRWISE_DISTANCE_FUNCTIONS。