Python - 並行不起作用

編輯：我在評論中得到了很好的反饋。對代碼做了一些更改，但仍然看到類似的問題。現在看來我的計時方式有些問題，因爲第二個操作需要超過0秒。Python - 並行不起作用

原文：我已經編寫了表面上並行的代碼，但實際上並沒有更快的運行 - 並行和非並行版本都花費相同的時間。對於我的生活，我無法弄清楚爲什麼。

我在Windows 7上通過Anaconda使用Python 3.4。無論我在IDE（Spyder）還是命令提示符下提交作業，結果都是一樣的。這裏是我的代碼：

import multiprocessing 
from multiprocessing import Pool 
import time 

def divide_by_two(n): 
    time.sleep(.1) 
    return n/2 

if __name__ == '__main__': 
    print("The number of cores is ", multiprocessing.cpu_count()) 
    pool = Pool(processes=multiprocessing.cpu_count()) 
    smallList = range(0,1000) #[0, 2, 4, 6, 8, 10] 
    print('Checking the parallelized way', smallList[:5]) 
    start = time.time() 
    result = pool.map(divide_by_two, smallList) 
    end = time.time() 
    cleaned = [x for x in result if not x is None] 
    print('small List divided by two is ', str(cleaned[:5])) 
    print('Parallel way takes ', str(end-start), ' seconds') 

    #Now the dumb version 
    print('Checking the slow way', smallList[:5]) 
    start2 = time.time() 
    smallList = range(0,1000) #[0, 2, 4, 6, 8, 10] 
    result2 = map(divide_by_two, smallList) 
    end2 = time.time() 
    cleaned2 = [x for x in result2 if not x is None] 
    print('small List divided by two is ', str(cleaned2[:5])) 
    print('The slow way takes', str(end2-start2), ' seconds')

這裏是輸出：

The number of cores is 4 
Checking the parallelized way range(0, 5) 
small List divided by two is [0.0, 0.5, 1.0, 1.5, 2.0] 
Parallel way takes 26.87681818008423 seconds 
Checking the slow way range(0, 5) 
small List divided by two is [0.0, 0.5, 1.0, 1.5, 2.0] 
The slow way takes 0.0 seconds

來源

2015-10-29 user3674422

在python中進行多處理時總會有小的開銷。當你爲了這麼小的任務運行它時，差異可以忽略不計。只有當你在更大的數據集上實現它時，你的速度纔會有所提高 – letsc

你的任務太小了。嘗試使用更耗時的功能，而不是睡眠。 – tuxtimo

似乎您在第二份打印聲明中使用了錯誤的「開始」時間。 – electrometro

感謝提意見。如果可以的話，我會盡快發表評論。並行化問題的大小增加了，這次使用了正確的變量（oops！）並使用了不同的定時函數。這裏是：

import multiprocessing 
from multiprocessing import Pool 
import time 

def divide_by_two(n): 
    time.sleep(.1) 
    return n/2 

if __name__ == '__main__': 
    print("The number of cores is ", multiprocessing.cpu_count()) 
    pool = Pool(processes=multiprocessing.cpu_count()) 
    smallList = range(0,1000) #[0, 2, 4, 6, 8, 10] 
    print('Checking the parallelized way', smallList[:5]) 
    start = time.clock() #time.time() 
    print('Start 1 is ', start) 
    result = pool.map(divide_by_two, smallList) 
    end = time.time() 
    cleaned = [x for x in result if not x is None] 
    print('small List divided by two is ', str(cleaned[:5])) 
    print('Parallel way takes ', time.clock()-start, ' seconds')  

    #Now the dumb version 
    print('Checking the slow way', smallList[:5]) 
    start = time.clock() 
    print('Start 2 is ', start) 
    result2 = map(divide_by_two, smallList) 
    cleaned2 = [x for x in result2 if not x is None] 
    print('small List divided by two is ', str(cleaned2[:5])) 
    print('The slow way takes', time.clock()-start, ' seconds')

來源

2015-10-29 19:25:35 user3674422

Python - 並行不起作用

回答

相關問題