2016-12-03 145 views
0

我嘗試使用python中的多處理讀取文件。這裏是一個小例子:Python多處理ThreadPool

import multiprocessing 
from time import * 

class class1(): 
    def function(self, datasheetname): 
     #here i start reading my datasheet 

if __name__ == '__main__': 
    #Test with multiprosessing 
    pool = multiprocessing.Pool(processes=4) 
    pool.map(class1("Datasheetname")) 
    pool.close() 

現在我得到以下錯誤:

TypeError: map() missing 1 required positional argument: 'iterable'

在這個板上的其他線程我得到了暗示與線程池要做到這一點,但我不無怎麼做。有任何想法嗎?

+0

你需要這樣做並行,或者你需要在一堆CSV/Excel表格的閱讀?如果後者可能使用[pandas.read_csv](http://pandas.pydata.org/pandas-docs/stable/generated/pandas.read_csv.html)或[pandas.read_excel](http:// pandas .pydata.org/pandas-docs/stable/generated/pandas.read_excel.html),它可以通過一次調用讀取多個文件/工作表。 – David

回答

2

Pool.map

map(func, iterable[, chunksize])

A parallel equivalent of the map() built-in function (it supports only one iterable argument though). It blocks until the result is ready.

This method chops the iterable into a number of chunks which it submits to the process pool as separate tasks. The (approximate) size of these chunks can be specified by setting chunksize to a positive integer.

您必須通過其中的每個元素被傳遞給目標func作爲每個過程參數可迭代。

例子:

def function(sheet): 
    # do something with sheet 
    return "foo" 

pool = Pool(processes=4) 
result = pool.map(function, ['sheet1', 'sheet2', 'sheet3', 'sheet4']) 
# result will be ['foo', 'foo', 'foo', 'foo']