2014-09-12 37 views
0
def multi(): 
    jobs = [] 
    r = open('raw.txt', 'r', 16777216).read().split('\n') 
    for i in r: 
     p = mp.Process(target=all, args=(i,)) 
     jobs.append(p) 
     p.start() 

raw.txt的每一行都是URL數據分割成塊,並添加號碼結果

請解釋我怎麼能修改多()來

一)分裂raw.txt成塊(就是說,每10行),並應用所有(),以每塊和處理的行

b)退回數量/塊到底

謝謝

+0

我會用一個發生器以你指定的任何數量的塊返回行。這樣做,你的其他要求是微不足道的 – 2014-09-12 14:10:14

回答

0

看看itertools包它有很多有用的東西。

>>> with open('input.txt', 'w') as f: 
... for i in xrange(998): 
...  f.write(uuid.uuid4().get_hex() + '\n') 
... 
>>> 
>>> from itertools import groupby, count  
>>> with open('input.txt', 'r') as f: 
...  samples = groupby(f, key=lambda k, line=count(): next(line)//100) 
...  for i in samples: 
...  print i 
... 
(0, <itertools._grouper object at 0x7f174f170c50>) 
(1, <itertools._grouper object at 0x7f1740804f50>) 
(2, <itertools._grouper object at 0x7f174f170c50>) 
(3, <itertools._grouper object at 0x7f1740804f50>) 
(4, <itertools._grouper object at 0x7f174f170c50>) 
(5, <itertools._grouper object at 0x7f1740804f50>) 
(6, <itertools._grouper object at 0x7f174f170c50>) 
(7, <itertools._grouper object at 0x7f1740804f50>) 
(8, <itertools._grouper object at 0x7f174f170c50>) 
(9, <itertools._grouper object at 0x7f1740804f50>)