如何在Python中一行一行地讀取文件（或stdin）而不是等待讀取整個文件

Question

我們在磁盤中有幾個大文件（大於RAM的大小）。我想在Python中逐行讀取它們並在終端中輸出結果。我經歷了[1]和[2]，但是我正在尋找不等到整個文件被讀入內存的方法。

我將利用這兩種命令：

cat fileName | python myScript1.py 
python myScript2.py fileName 

[1] How do you read from stdin in Python? [2] How do I write a unix filter in python?

Answer 1

這是在Python standard behavior of file objects：

with open("myfile.txt", "r") as myfile: 
    for line in myfile: 
     # do something with the current line 

或

for line in sys.stdin: 
    # do something with the current line 

Answer 2

只要迭代file：

with open('huge.file') as hf: 
    for line in hf: 
    if 'important' in line: 
     print(line) 

這將需要ö （1）記憶。

從標準輸入讀取，只需在sys.stdin而不是hf迭代：

import sys 
for line in sys.stdin: 
    if 'important' in line: 
    print(line) 

Answer 3

if __name__ == '__main__': 
    while 1: 
     try: 
      a=raw_input() 
     except EOFError: 
      break 
     print a 

這將從stdin直到EOF讀取。 要使用第二種方法讀取一個文件，你可以用添的方法

即

with open("myfile.txt", "r") as myfile: 
    for line in myfile: 
     print line 
     # do something with the current line