使用python如何同時爲多個文件執行此代碼？

Question

我想要同時拖放多個文件並將日誌推到劃線上。 我正在從配置文件中讀取文件，然後我想要拖尾每個文件並將日誌發送給抄寫員。 我試過的是發送日誌只有第一個文件，而不是其他人。

我想同時爲每個文件運行拖尾並同時爲它們中的每一個發送日誌。

for l in Config.items('files'): 
    print l[0] 
    print l[1] 
    filename = l[1] 
    file = open(filename,'r') 
    st_results = os.stat(l[1]) 
    st_size = st_results[6] 
    file.seek(st_size) 
    while 1: 
    where = file.tell() 
    line = file.readline() 
    if not line: 
     time.sleep(1) 
     file.seek(where) 
    else: 
     print line, # already has newline 
     category=l[0] 
     message=line 
     log_entry = scribe.LogEntry(category, message) 
     socket = TSocket.TSocket(host='localhost', port=1463) 
     transport = TTransport.TFramedTransport(socket) 
     protocol = TBinaryProtocol.TBinaryProtocol(trans=transport, strictRead=False, strictWrite=False) 
     client = scribe.Client(iprot=protocol, oprot=protocol) 
     transport.open() 
     result = client.Log(messages=[log_entry]) 
     transport.close() 

Answer 1

不同的實現的@Pengman's idea：

#!/usr/bin/env python 
import os 
import time 
from threading import Thread 

def follow(filename): 
    with open(filename) as file: 
     file.seek(0, os.SEEK_END) # goto EOF 
     while True: 
      for line in iter(file.readline, ''): 
       yield line 
      time.sleep(1) 

def logtail(category, filename): 
    print category 
    print filename 
    for line in follow(filename): 
     print line, 
     log_entry(category, line) 

for args in Config.items('files'): 
    Thread(target=logtail, args=args).start() 

哪裏log_entry()是從問題的代碼的副本：

def log_entry(category, message): 
    entry = scribe.LogEntry(category, message) 
    socket = TSocket.TSocket(host='localhost', port=1463) 
    transport = TTransport.TFramedTransport(socket) 
    protocol = TBinaryProtocol.TBinaryProtocol(trans=transport,strictRead=False, 
               strictWrite=False) 
    client = scribe.Client(iprot=protocol, oprot=protocol) 
    transport.open() 
    result = client.Log(messages=[entry]) 
    transport.close() 

follow()可以使用FS監視工具來實現，參見tail -f in python with no time.sleep。

Answer 2

嘗試這樣的事情(Inspired by this)

import threading 

def monitor_file(l): 

    print l[0] 
    print l[1] 
    filename = l[1] 
    file = open(filename,'r') 
    st_results = os.stat(l[1]) 
    st_size = st_results[6] 
    file.seek(st_size) 
    while 1: 
     where = file.tell() 
     line = file.readline() 
     if not line: 
     time.sleep(1) 
     file.seek(where) 
     else: 
     print line, # already has newline 
     category=l[0] 
     message=line 
     log_entry = scribe.LogEntry(category, message) 
     socket = TSocket.TSocket(host='localhost', port=1463) 
     transport = TTransport.TFramedTransport(socket) 
     protocol = TBinaryProtocol.TBinaryProtocol(trans=transport, strictRead=False,  strictWrite=False) 
     client = scribe.Client(iprot=protocol, oprot=protocol) 
     transport.open() 
     result = client.Log(messages=[log_entry]) 
     transport.close() 


for l in Config.items('files'): 
    thread = threading.Thread(target=monitor_file, args=(l))