urllib2 POST進度監控

Question

我正在通過POST將一個相當大的文件與urllib2上傳到服務器端腳本。我想顯示一個顯示當前上傳進度的進度指示器。是否有urllib2提供的鉤子或回調允許我監視上傳進度？我知道你可以通過連續調用read（）方法進行下載，但是我沒有看到write（）方法，只是將數據添加到請求中。

Answer 1

這是可能的，但你需要做的幾件事情：

• 僞造出來的urllib2的子系統爲傳遞文件句柄向下至httplib的通過附加__len__屬性，這使得len(data)返回正確的尺寸，用於填充Content-Length標頭。
• 覆蓋文件句柄上的read()方法：由於httplib調用read()您的回調將被調用，讓您計算百分比並更新您的進度欄。

  import os, urllib2 
  from cStringIO import StringIO 
  
  class Progress(object): 
      def __init__(self): 
       self._seen = 0.0 
  
      def update(self, total, size, name): 
       self._seen += size 
       pct = (self._seen/total) * 100.0 
       print '%s progress: %.2f' % (name, pct) 
  
  class file_with_callback(file): 
      def __init__(self, path, mode, callback, *args): 
       file.__init__(self, path, mode) 
       self.seek(0, os.SEEK_END) 
       self._total = self.tell() 
       self.seek(0) 
       self._callback = callback 
       self._args = args 
  
      def __len__(self): 
       return self._total 
  
      def read(self, size): 
       data = file.read(self, size) 
       self._callback(self._total, len(data), *self._args) 
       return data 
  
  path = 'large_file.txt' 
  progress = Progress() 
  stream = file_with_callback(path, 'rb', progress.update, path) 
  req = urllib2.Request(url, stream) 
  res = urllib2.urlopen(req) 
  

  輸出：

  large_file.txt progress: 0.68 
  large_file.txt progress: 1.36 
  large_file.txt progress: 2.04 
  large_file.txt progress: 2.72 
  large_file.txt progress: 3.40 
  ... 
  large_file.txt progress: 99.20 
  large_file.txt progress: 99.87 
  large_file.txt progress: 100.00 
  

這可以與任何類似文件的對象的工作，但我已經包裹file顯示它如何能與一個真正的大文件，從工作流盤

Answer 2

我不認爲這是可能的，但你可以使用pycurl does have upload/download progress callbacks。

Answer 3

requests 2.0.0 has streaming uploads。這意味着您可以使用生成器生成小塊並打印塊之間的進度。

Answer 4

poster支持此

import json 
import os 
import sys 
import urllib2 

from poster.encode import multipart_encode 
from poster.streaminghttp import register_openers 

def _upload_progress(param, current, total): 
    sys.stdout.write(
     "\r{} - {:.0f}%    " 
     .format(param.name, 
       (float(current)/float(total)) * 100.0)) 
    sys.stdout.flush() 

def upload(request_resource, large_file_path): 
    register_openers() 
    with open(large_file_path, 'r') as large_file: 
     request_data, request_headers = multipart_encode(
      [('file', largs_file)], 
      cb=_upload_progress) 

     request_headers.update({ 
      'X-HockeyAppToken': 'we use this for hockeyapp upload' 
     }) 

     upload_request = urllib2.Request(request_resource, 
             request_data, 
             request_headers) 
     upload_connection = urllib2.urlopen(upload_request) 
     upload_response = json.load(upload_connection) 
    print "Done"