保存gzipped JSON導致連接重置

我有一個gzipped JSON file，我嘗試下載，但是當我嘗試讀取請求庫的響應內容時，它會重置連接。保存gzipped JSON導致連接重置

data = requests.request("GET", i, stream=True) 
with gzip.open(i.rsplit("/")[-1], "wb") as fh: 
    for chunk in data.iter_content(chunk_size=1024): 
    fh.write(chunk)

這裏是我所得到的，當我嘗試閱讀的內容：

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 228, in _error_catcher 
    yield 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 501, in read_chunked 
    chunk = self._handle_chunk(amt) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 461, in _handle_chunk 
    value = self._fp._safe_read(amt) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\http\client.py", line 592, in _safe_read 
    chunk = self.fp.read(min(amt, MAXAMOUNT)) 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\socket.py", line 575, in readinto 
    return self._sock.recv_into(b) 
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 664, in generate 
    for chunk in self.raw.stream(chunk_size, decode_content=True): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 349, in stream 
    for line in self.read_chunked(amt, decode_content=decode_content): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 526, in read_chunked 
    self._original_response.close() 
    File "C:\Users\Mike\AppData\Local\Programs\Python\Python35\Lib\contextlib.py", line 77, in __exit__ 
    self.gen.throw(type, value, traceback) 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\packages\urllib3\response.py", line 246, in _error_catcher 
    raise ProtocolError('Connection broken: %r' % e, e) 
requests.packages.urllib3.exceptions.ProtocolError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)) 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "H:/Programming/Python/warehouse/main.py", line 55, in <module> 
    compile_auctions(slugs) 
    File "H:/Programming/Python/warehouse/main.py", line 44, in compile_auctions 
    for chunk in data.iter_content(chunk_size=1024): 
    File "H:\Programming\Python\virtualenvs\warehouse\lib\site-packages\requests\models.py", line 667, in generate 
    raise ChunkedEncodingError(e) 
requests.exceptions.ChunkedEncodingError: ("Connection broken: ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None)", ConnectionResetError(10054, 'An existing connection was forcibly closed by the remote host', None, 10054, None))

我可以在我的瀏覽器，在郵差閱讀JSON，所以我不知道我在做什麼錯誤。我在Windows 10上使用Python 3.5.1。

這是保存gzip JSON的正確方法嗎？

來源

2016-06-12 mxplusb

請求已經解壓縮數據。

你並不需要用gzip：

import requests 

req = requests.get("http://auction-api-us.worldofwarcraft.com/auction-data/4923213e5eb3ec3b7e03d22b632bda36/auctions.json", stream=True) 

with open("out.json", "wb") as f: 
    for chunk in req.iter_content(chunk_size=4096): 
     f.write(chunk)

來源

2016-06-12 15:53:47

在我的機器上，按預期工作。主機是否因爲太多請求而將您列入黑名單？編輯：這是一個回覆刪除評論 –

事實證明，這個問題是別的東西，而不是我的代碼。不過謝謝！ – mxplusb

@mynameismevin你能透露是什麼問題？我正在努力解決同樣的錯誤。 – Adam

所以，問題不在於實際上我使用的API，它要麼與請求庫for Windows或問題與Python 3.5的錯誤.1在Windows 10上。

當我使用Python 3.5.1在Ubuntu 16.04上運行這個確切的代碼時，它工作得很好。

來源

2016-06-13 15:03:35 mxplusb

保存gzipped JSON導致連接重置

回答

相關問題