Python：用islice一次讀取N個行數的問題

Question

我正嘗試使用「from itertools import islice」以便從使用liblas模塊的* .las文件中一次讀取多行代碼。 （我的目標是閱讀的塊狀bychunk）

以下問題：Python how to read N number of lines at a time

islice() can be used to get the next n items of an iterator. Thus, list(islice(f, n)) will return a list of the next n lines of the file f. Using this inside a loop will give you the file in chunks of n lines. At the end of the file, the list might be shorter, and finally the call will return an empty list.

我用下面的代碼：

from numpy import nonzero 
from liblas import file as lasfile 
from itertools import islice 


chunkSize = 1000000 

f = lasfile.File(inFile,None,'r') # open LAS 
while True: 
    chunk = list(islice(f,chunkSize)) 
    if not chunk: 
     break 
    # do other stuff 

，但我有這個問題：

len(f) 
2866390 

chunk = list(islice(f, 1000000)) 
len(chunk) 
**1000000** 
chunk = list(islice(f, 1000000)) 
len(chunk) 
**1000000** 
chunk = list(islice(f, 1000000)) 
len(chunk) 
**866390** 
chunk = list(islice(f, 1000000)) 
len(chunk) 
**1000000** 

當文件f到達時，islice重新開始讀取文件。

感謝您的任何建議和幫助。這是非常欣賞

Answer 1

現在看來似乎會很容易足夠寫一發生器同時產生n行：

def n_line_iterator(fobj,n): 
    if n < 1: 
     raise ValueError("Must supply a positive number of lines to read") 

    out = [] 
    num = 0 
    for line in fobj: 
     if num == n: 
      yield out #yield 1 chunk 
      num = 0 
      out = [] 
     out.append(line) 
     num += 1 
    yield out #need to yield the rest of the lines 

Answer 2

變化file.py屬於liblas包的源代碼。目前__iter__定義爲（src on github）

def __iter__(self): 
    """Iterator support (read mode only) 

     >>> points = [] 
     >>> for i in f: 
     ... points.append(i) 
     ... print i # doctest: +ELLIPSIS 
     <liblas.point.Point object at ...> 
    """ 
    if self.mode == 0: 
     self.at_end = False 
     p = core.las.LASReader_GetNextPoint(self.handle) 
     while p and not self.at_end: 
      yield point.Point(handle=p, copy=True) 
      p = core.las.LASReader_GetNextPoint(self.handle) 
      if not p: 
       self.at_end = True 
     else: 
      self.close() 
      self.open() 

你看，當文件是在結束其關閉並再次打開，所以迭代在文件的重新開始。

嘗試在一段時間後刪除最後一個else塊，所以該方法的正確的代碼應該是：

def __iter__(self): 
    """Iterator support (read mode only) 

     >>> points = [] 
     >>> for i in f: 
     ... points.append(i) 
     ... print i # doctest: +ELLIPSIS 
     <liblas.point.Point object at ...> 
    """ 
    if self.mode == 0: 
     self.at_end = False 
     p = core.las.LASReader_GetNextPoint(self.handle) 
     while p and not self.at_end: 
      yield point.Point(handle=p, copy=True) 
      p = core.las.LASReader_GetNextPoint(self.handle) 
      if not p: 
       self.at_end = True