Python subprocess.popen和rdiff-backup

我想在python中創建一個rdiff-backup包裝程序，用於將windows機器備份到Linux服務器。Python subprocess.popen和rdiff-backup

我想在包裝程序中處理rdiff-backup的輸出。但是，當使用子流程模塊和管道stdout和stderr對包裝器執行rdiff-backup時，stdout始終會結束管道的末端。

當不使用管道輸出標準輸出的情況下使用子進程時，控制檯中按照正確順序打印標準錯誤。

當我在本地使用rdiff-backup而沒有使用ssh管道時，stdout和stderr的順序是正確的。 Rdiff-backup還使用subprocess.popen打開ssh會話並將數據傳輸到服務器。我懷疑由於某種原因stdout被阻塞，直到ssh會話關閉。

這裏是我的代碼，這是一個簡化版本，真正的程序使用線程監聽到stdout：

import sys 
import subprocess 

class Rdiffbackup(object): 
    def __init__(self): 
     #self.io_q = Queue() 
     self.exe = 'F:\\workspace\\pysubprocess\\bin\\rdiff-backup\\rdiff-backup.exe' 

     self.verbosity = '-v5' 

     self.ssh_exe = './bin/openssh/bin/ssh' 
     self.quiet = '-q' 
     self.compression = '-C' 
     self.port = '-p 5555' 
     self.key = '-i ./keys/rdiffbackup' 
     self.options = '-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null' 
     self.remote_schema = self.ssh_exe + ' ' + self.quiet + ' ' + self.compression + ' ' + self.port + ' ' + self.key + ' ' + self.options + ' %s rdiff-backup --server' 

    def start(self,source,dest): 
     with subprocess.Popen([self.exe,self.verbosity,'--remote-schema',self.remote_schema,source,dest],stdout=subprocess.PIPE,stderr=subprocess.STDOUT) as self.proc:  
      for line in self.proc.stdout: 
       sys.stdout.write(line.decode("utf-8"))  

if __name__ == '__main__': 
    rdb = Rdiffbackup() 
    source = "C:/Users/vdrmrt/Desktop/data"  
    dest = "[email protected]::backup" 
    rdb.start(source,dest)

輸出：

----------------------------------------------------------------- 
Detected abilities for source (read only) file system: 
    Access control lists       Off 
    Extended attributes       Off 
    Windows access control lists     On 
    Case sensitivity        Off 
    Escape DOS devices       Off 
    Escape trailing spaces      Off 
    Mac OS X style resource forks    Off 
    Mac OS X Finder information     Off 
----------------------------------------------------------------- 
Unable to import win32security module. Windows ACLs 
not supported by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0 
escape_dos_devices not required by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0 
----------------------------------------------------------------- 
Detected abilities for destination (read/write) file system: 
    Ownership changing       Off 
    Hard linking         On 
    fsync() directories       On 
    Directory inc permissions     On 
    High-bit permissions       On 
    Symlink permissions       Off 
    Extended filenames       On 
    Windows reserved filenames     Off 
    Access control lists       On 
    Extended attributes       On 
    Windows access control lists     Off 
    Case sensitivity        On 
    Escape DOS devices       Off 
    Escape trailing spaces      Off 
    Mac OS X style resource forks    Off 
    Mac OS X Finder information     Off 
----------------------------------------------------------------- 
Backup: must_escape_dos_devices = 0 
Using rdiff-backup version 1.2.8 
Executing ./bin/openssh/bin/ssh -q -C -p 5555 -i ./keys/rdiffbackup -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null [email protected] rdiff-backup --server 
Hardlinks disabled by default on Windows 
Unable to import module xattr. 
Extended attributes not supported on filesystem at C:/Users/vdrmrt/Desktop/data 
Unable to import module posix1e from pylibacl package. 
POSIX ACLs not supported on filesystem at C:/Users/vdrmrt/Desktop/data 
escape_dos_devices not required by filesystem at C:/Users/vdrmrt/Desktop/data 
Symbolic links excluded by default on Windows 
Starting increment operation C:/Users/vdrmrt/Desktop/data to backup

正確的輸出：

Using rdiff-backup version 1.2.8 
Executing ./bin/openssh/bin/ssh -q -C -p 5555 -i ./keys/rdiffbackup -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null [email protected] rdiff-backup --server 
Hardlinks disabled by default on Windows 
Unable to import module xattr. 
Extended attributes not supported on filesystem at C:/Users/vdrmrt/Desktop/data 
Unable to import module posix1e from pylibacl package. 
POSIX ACLs not supported on filesystem at C:/Users/vdrmrt/Desktop/data 
escape_dos_devices not required by filesystem at C:/Users/vdrmrt/Desktop/data 
----------------------------------------------------------------- 
Detected abilities for source (read only) file system: 
    Access control lists       Off 
    Extended attributes       Off 
    Windows access control lists     On 
    Case sensitivity        Off 
    Escape DOS devices       Off 
    Escape trailing spaces      Off 
    Mac OS X style resource forks    Off 
    Mac OS X Finder information     Off 
----------------------------------------------------------------- 
Unable to import win32security module. Windows ACLs 
not supported by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0 
escape_dos_devices not required by filesystem at backup/rdiff-backup-data/rdiff-backup.tmp.0 
----------------------------------------------------------------- 
Detected abilities for destination (read/write) file system: 
    Ownership changing       Off 
    Hard linking         On 
    fsync() directories       On 
    Directory inc permissions     On 
    High-bit permissions       On 
    Symlink permissions       Off 
    Extended filenames       On 
    Windows reserved filenames     Off 
    Access control lists       On 
    Extended attributes       On 
    Windows access control lists     Off 
    Case sensitivity        On 
    Escape DOS devices       Off 
    Escape trailing spaces      Off 
    Mac OS X style resource forks    Off 
    Mac OS X Finder information     Off 
----------------------------------------------------------------- 
Backup: must_escape_dos_devices = 0 
Symbolic links excluded by default on Windows 
Starting increment operation C:/Users/vdrmrt/Desktop/data to backup

來源

2013-08-18 vdrmrt

我終於能夠解決這個問題。
Rdiff-backup緩衝stdout和stderr，當它被連接到其他程序時。解決的辦法是重建rdiff進行備份一個額外的py2exe選項：「緩衝」：真
我也不得不增加額外的選項，以使與Windows 7
Py2exe - win32api.pyc ImportError DLL load failed

這裏的建築時，它的工作來自rdiff-backup構建的最終setup.py的片段。

if '--single-file' in sys.argv[1:]: 
      sys.argv.remove('--single-file') 
      extra_options.update({ 
       'options': {'py2exe': {'bundle_files': 1, 
             'unbuffered': True, 
             'dll_excludes': [ "mswsock.dll", "powrprof.dll" ]}}, 
       'zipfile': None 
      })

來源

2013-08-25 09:27:13 vdrmrt

嘗試使用單獨的管道stdout和stderr。首先閱讀stdout行，然後閱讀stderr。

來源

2013-08-18 12:53:45

已經嘗試過。通常rdiff-backup首先有一些stdout，然後是stderr，然後再次stdout。當我單獨在線程中讀取它們時，我可以在程序運行時處理它們，標準輸出只在最後填充。 – vdrmrt

@vdrmrt然後保存stderr輸出，直到stdout完成，然後再打印。 –

我想在rdiff運行時用輸出做東西。 – vdrmrt

Python subprocess.popen和rdiff-backup

回答

相關問題