Python glob.glob（dir）內存錯誤

Question

我在處理數百萬個文件搜索文件夾時遇到內存問題。有誰知道如何克服這種情況？有沒有一些方法來限制glob將搜索的文件？所以它可以分塊執行？

Traceback (most recent call last): 
    File "./lb2_lmanager", line 533, in <module> 
    main(sys.argv[1:]) 
File "./lb2_lmanager", line 318, in main 
    matched = match_files(policy.directory, policy.file_patterns) 
    File "./lb2_lmanager", line 32, in wrapper 
    res = func(*args, **kwargs) 
    File "./lb2_lmanager", line 380, in match_files 
    listing = glob.glob(directory) 
    File "/usr/lib/python2.6/glob.py", line 16, in glob 
    return list(iglob(pathname)) 
    File "/usr/lib/python2.6/glob.py", line 43, in iglob 
    yield os.path.join(dirname, name) 
    File "/usr/lib/python2.6/posixpath.py", line 70, in join 
    path += '/' + b 
MemoryError 

Answer 1

嘗試使用generators而不是lists。
要了解發電機讀this

import glob 
dir_list = glob.iglob(YOUR_DIRECTORY) 
for file in dir_list: 
    print file 

變化YOUR_DIRECTORY到您想列出的目錄。