Python請求：在連接到代理服務器時忽略異常和錯誤

Question

我用Python編寫了我的第一個程序。

#This program casts votes in online poll using different proxy servers for each request. 
#It works, but some proxy servers cause errors crashing the whole thing. 
#To avoid that, I would like it to skip those servers and ignore the errors. 

import requests 
import time 

#Votes to be cast 
votes = 5 

#Makes proxy list 
f=open('proxy2.txt') 
lines=f.read().splitlines() 
f.close() 

#Vote counter 
i = 1 

#Proxy list counter 
j = 0 

while (i<=votes): 

#Tests and moves to next proxy if there was a problem. 
    try: 
     r = requests.get('http://www.google.com') 
    except requests.exceptions.RequestException: 
     j = j + 1 

#Headers copied from my browser. Some of them cause errors. Could you tell me why? 
    headers = { 
        'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8', 
        #'Accept-Encoding': 'gzip, deflate', 
        #'Accept-Language': 'pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4', 
        #'Cache-Control': 'max-age=0', 
        #'Connection': 'keep-alive', 
        #'Content-Length': '101', 
        'Content-Type': 'application/x-www-form-urlencoded', 
        #'Host': 'www.mylomza.pl', 
        #'Origin': 'http://www.mylomza.pl', 
        #'Referer': 'http://www.mylomza.pl/home/lomza/item/11780-wybierz-miss-%C5%82ks-i-portalu-mylomzapl-video-i-foto.html', 
        #'Upgrade-Insecure-Requests': '1', 
        #'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.102 Safari/537.36' 
       } 

    proxies = { 
        'http': 'http://'+lines[j] #31.207.0.99:3128 
       } 

    r = requests.get('http://www.mylomza.pl/home/lomza/item/11780-wybierz-miss-%C5%82ks-i-portalu-mylomzapl-video-i-foto.html', headers=headers, proxies=proxies, timeout=10) 

#The funny part - form, that I have to post, requires some kind of ID and this is my way of getting it :P Feel free to suggest an alternative way. 
    userid = r.text[(22222-32):22222] 

    print('Voter', userid, 'registered.') 

    data = { 
       'voteid': '141', 
       'task_button': 'Głosuj', 
       'option': 'com_poll', 
       'task': 'vote', 
       'id': '25', 
       userid: '1' 
       } 

    r = requests.post('http://www.mylomza.pl/home/lomza/item/index.php', headers=headers, cookies=r.cookies, data=data, proxies=proxies, timeout=10) 

    print('Vote nr', i, 'cast from', lines[i]) 

    i = i + 1 
    j = j + 1 

    time.sleep(1) 

我需要的是讓它處理異常和錯誤。

#Tests and moves to next proxy if there was a problem. 
try: 
    r = requests.get('http://www.google.com') 
except requests.exceptions.RequestException: 
    j = j + 1 

旁邊，我可以用實現這一目標的另一種方式：

#The funny part - form, that I have to post, requires some kind of ID and this is my way of getting it :P Feel free to suggest an alternative way. 
    userid = r.text[(22222-32):22222] 

有時候，我的方法是行不通的（如下圖所示）。第一次投票通過，第二次沒有，然後全部墜毀。

Voter 53bf55490ebd07d9c190787c5c6ca44c registered. 
Vote nr 1 cast from 111.23.6.161:80 
Voter registered. 
Vote nr 2 cast from 94.141.102.203:8080 
Traceback (most recent call last): 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 142, in _new_conn 
    (self.host, self.port), self.timeout, **extra_kw) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\connection.py", line 91, in create_connection 
    raise err 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\connection.py", line 81, in create_connection 
    sock.connect(sa) 
socket.timeout: timed out 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 578, in urlopen 
    chunked=chunked) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 362, in _make_request 
    conn.request(method, url, **httplib_request_kw) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1083, in request 
    self._send_request(method, url, body, headers) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1128, in _send_request 
    self.endheaders(body) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 1079, in endheaders 
    self._send_output(message_body) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 911, in _send_output 
    self.send(msg) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\http\client.py", line 854, in send 
    self.connect() 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 167, in connect 
    conn = self._new_conn() 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connection.py", line 147, in _new_conn 
    (self.host, self.timeout)) 
requests.packages.urllib3.exceptions.ConnectTimeoutError: (<requests.packages.urllib3.connection.HTTPConnection object at 0x03612730>, 'Connection to 94.141.102.203 timed out. (connect timeout=10)') 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 403, in send 
    timeout=timeout 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 623, in urlopen 
    _stacktrace=sys.exc_info()[2]) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\packages\urllib3\util\retry.py", line 281, in increment 
    raise MaxRetryError(_pool, url, error or ResponseError(cause)) 
requests.packages.urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='94.141.102.203', port=8080): Max retries exceeded with url: http://www.mylomza.pl/home/lomza/item/11780-wybierz-miss-%C5%82ks-i-portalu-mylomzapl-video-i-foto.html (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x03612730>, 'Connection to 94.141.102.203 timed out. (connect timeout=10)')) 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\PollVoter.py", line 50, in <module> 
    r = requests.get('http://www.mylomza.pl/home/lomza/item/11780-wybierz-miss-%C5%82ks-i-portalu-mylomzapl-video-i-foto.html', headers=headers, proxies=proxies, timeout=10) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 71, in get 
    return request('get', url, params=params, **kwargs) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\api.py", line 57, in request 
    return session.request(method=method, url=url, **kwargs) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 475, in request 
    resp = self.send(prep, **send_kwargs) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\sessions.py", line 585, in send 
    r = adapter.send(request, **kwargs) 
    File "C:\Users\Adrian\AppData\Local\Programs\Python\Python35-32\lib\site-packages\requests\adapters.py", line 459, in send 
    raise ConnectTimeout(e, request=request) 
requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='94.141.102.203', port=8080): Max retries exceeded with url: http://www.mylomza.pl/home/lomza/item/11780-wybierz-miss-%C5%82ks-i-portalu-mylomzapl-video-i-foto.html (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x03612730>, 'Connection to 94.141.102.203 timed out. (connect timeout=10)')) 

Answer 1

問題解決了

原來，我不應該打開每個代理服務器超過1個連接。我不得不提出2個請求。解決方案是從我的IP發送第一個，然後切換到第二個代理。

r = requests.get(url, headers=headers, timeout=timeout) 

try: 
    r = requests.post(url, headers=headers, cookies=r.cookies, data=data, timeout=timeout, proxies=proxies) 
except: 
     j = j + 1 

到目前爲止工作完美。 :)

Answer 2

它看起來像你充斥着太多的請求的服務器，這就是爲什麼你得到其他錯誤，如requests.packages.urllib3.exceptions.MaxRetryError，因爲有可能是服務器節流，你可以在給定的量使連接的數量時間。您可以嘗試處理輸出中列出的所有例外情況，並且還可以嘗試對您請求的網址嘗試進行更少的嘗試。

[編輯]或者，如果你想蠻力和處理所有的錯誤和異常，請嘗試以下，而不是

except: 
    j = j + 1 

[編輯：]你可以嘗試https:以及http:

[編輯]發現此：

如果遠程服務器速度很慢，您可以通過將None作爲超時值傳遞，然後檢索ac咖啡的特寫。

r = requests.get('https://github.com', timeout=None) 

Answer 3

我有類似的事情，並使用

except: 
    continue 

其中彪又繼續循環遍歷異常的情況下，繼續「努力」