Python numpy.fft在子進程中運行時非常慢（10倍慢）

Question

我發現numpy.fft.fft（及其變體）在後臺運行時非常慢。下面是一個例子就是我在談論

import numpy as np 
import multiprocessing as mproc 
import time 
import sys 

# the producer function, which will run in the background and produce data 
def Producer(dataQ): 
    numFrames = 5 
    n = 0 
    while n < numFrames: 
     data = np.random.rand(3000, 200) 
     dataQ.put(data) # send the datta to the consumer 
     time.sleep(0.1) # sleep for 0.5 second, so we dont' overload CPU 
     n += 1    

# the consumer function, which will run in the backgrounnd and consume data from the producer 
def Consumer(dataQ): 
    while True: 
     data = dataQ.get() 
     t1 = time.time() 
     fftdata = np.fft.rfft(data, n=3000*5) 
     tDiff = time.time() - t1 
     print("Elapsed time is %0.3f" % tDiff) 
     time.sleep(0.01) 
     sys.stdout.flush() 

# the main program if __name__ == '__main__': is necessary to prevent this code from being run 
# only when this program is started by user 
if __name__ == '__main__':  
    data = np.random.rand(3000, 200) 
    t1 = time.time() 
    fftdata = np.fft.rfft(data, n=3000*5, axis=0) 
    tDiff = time.time() - t1 
    print("Elapsed time is %0.3f" % tDiff) 

    # generate a queue for transferring data between the producedr and the consumer 
    dataQ = mproc.Queue(4) 

    # start up the processoso 
    producerProcess = mproc.Process(target=Producer, args=[dataQ], daemon=False) 
    consumerProcess = mproc.Process(target=Consumer, args=[dataQ], daemon=False) 
    print("starting up processes") 

    producerProcess.start() 
    consumerProcess.start() 
    time.sleep(10) # let program run for 5 seconds 

    producerProcess.terminate() 
    consumerProcess.terminate() 

我的機器上它produes輸出：在後臺運行時

Elapsed time is 0.079 
starting up processes 
Elapsed time is 0.859 
Elapsed time is 0.861 
Elapsed time is 0.878 
Elapsed time is 0.863 
Elapsed time is 0.758 

正如你所看到的，它是大約慢10倍，和我無法弄清楚爲什麼會出現這種情況。 time.sleep（）調用應確保其他進程（主進程和生產者進程）在計算FFT時不做任何事情，因此它應該使用所有內核。我已經通過Windows任務管理器檢查了CPU使用率，並且在單進程和多進程情況下調用numpy.fft.fft時似乎佔用了大約25％。

任何人都有一個想法是怎麼回事？

Answer 1

的主要問題是，在後臺線程的FFT電話是：

fftdata = np.fft.rfft(data, n=3000*5) 

而不是：

fftdata = np.fft.rfft(data, n=3000*5, axis=0) 

這對我來說一切都不同。

還有其他一些值得注意的事情。爲什麼不讓處理器自己處理這個問題，而不是到處都有time.sleep()？更進一步，而不是暫停主線程，你可以使用

consumerProcess.join() 

，然後有生產者進程運行dataQ.put(None)一旦完成加載數據，並打破了在消費過程中的循環，即：

def Consumer(dataQ): 
    while True: 
     data = dataQ.get() 
     if(data is None): 
      break 
     ...