使用urllib登錄到網站Python 3.x

我正嘗試以編程方式使用urllib登錄到Myanimelist.net。我不確定發生了什麼事。我沒有正確登錄，因爲它返回的代碼不是我所期望的。不知道POST參數應該是什麼或者我做錯了什麼。看着一堆關於urllib登錄認證的類似的stackoverflow問題，但我無法弄清楚。使用urllib登錄到網站Python 3.x

這是形式：

<form action="http://myanimelist.net/login.php" id="loginForm" method="post" name="loginForm"> 
<table align="center" border="0" cellpadding="6" cellspacing="0"> 
    <tbody> 
     <tr> 
      <td width="100"><strong>Username:</strong></td> 
      <td> 
       <input class="inputtext" id="loginUNAME" name="username"size="30" type="text" value=""> 
      </td> 
     </tr> 
     <tr> 
      <td><strong>Password:</strong></td> 
      <td> 
       <input class="inputtext" name="password" size="30" type="password"> 
      </td> 
     </tr> 
     <tr> 
      <td align="center" colspan="2"> 
       <input name="cookie" type="checkbox" value="1"> Always stay logged in? 
      </td> 
     </tr> 
     <tr> 
      <td align="center" colspan="2"><input class="inputButton" name= "sublogin" type="submit" value="Login">&nbsp;&nbsp; 
      <input class="inputButton" name="register" onclick="document.location='http://myanimelist.net/register.php';" type="button" value="Register"></td> 
     </tr> 
     <tr> 
      <td align="center" colspan="2"> 
       <a href="http://myanimelist.net/password.php">Forget Your 
       Password?</a> 
      </td> 
     </tr> 
    </tbody> 
</table> 
</form>

我的登錄代碼：

import http.cookiejar 
import urllib 
from bs4 import BeautifulSoup 

cj = http.cookiejar.CookieJar() 
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj)) 

opener.addheaders = [('User-agent', 'Mozilla/5.0')] 

urllib.request.install_opener(opener) 

url = 'http://myanimelist.net/login.php' 

payload = { 
'username' : '<username>', 
'password' : '<password>', 
'cookie' : '1', 
'sublogin' : 'Login' 
} 

data = urllib.parse.urlencode(payload).encode('ascii') 

request = urllib.request.Request(url=url, data=data) 

response = urllib.request.urlopen(request) 

html = response.read().decode('utf-8') 

soup = BeautifulSoup(html) 

print(soup.prettify())

什麼，我得到的是：

<html> 
<head> 
<META NAME="robots" CONTENT="noindex,nofollow"> 
<script> 
(function(){function getSessionCookies(){cookieArray=new Array();var cName=/^\s? 
incap_ses_/;var c=document.cookie.split(";");for(var i=0;i<c.length;i++){key=c[i 
].substr(0,c[i].indexOf("="));value=c[i].substr(c[i].indexOf("=")+1,c[i].length) 
;if(cName.test(key)){cookieArray[cookieArray.length]=value}}return cookieArray}f 
unction setIncapCookie(vArray){try{cookies=getSessionCookies();digests=new Array 
(cookies.length);for(var i=0;i<cookies.length;i++){digests[i]=simpleDigest((vArr 
ay)+cookies[i])}res=vArray+",digest="+(digests.join())}catch(e){res=vArray+",dig 
est="+(encodeURIComponent(e.toString()))}createCookie("___utmvc",res,20)}functio 
n simpleDigest(mystr){var res=0;for(var i=0;i<mystr.length;i++){res+=mystr.charC 
odeAt(i)}return res}function createCookie(name,value,seconds){if(seconds){var da 
te=new Date();date.setTime(date.getTime()+(seconds*1000));var expires="; expires 
="+date.toGMTString()}else{var expires=""}document.cookie=name+"="+value+expires 
+"; path=/"}function test(o){var res="";var vArray=new Array();for(test in o){sw 
itch(o[test]){case"exists":try{vArray[vArray.length]=encodeURIComponent(test+"=" 
+typeof(eval(test)))}catch(e){vArray[vArray.length]=encodeURIComponent(test+"="+ 
e)}break;case"value":try{vArray[vArray.length]=encodeURIComponent(test+"="+eval(
test).toString())}catch(e){vArray[vArray.length]=encodeURIComponent(test+"="+e)} 
break;case"plugins":try{p=navigator.plugins;pres="";for(a in p){pres+=(p[a]["des 
cription"]+" ").substring(0,20)}vArray[vArray.length]=encodeURIComponent("plugin 
s="+pres)}catch(e){vArray[vArray.length]=encodeURIComponent("plugins="+e)}break; 
case"plugin":try{a=navigator.plugins;for(i in a){f=a[i]["filename"].split(".");i 
f(f.length==2){vArray[vArray.length]=encodeURIComponent("plugin="+f[1]);break}}} 
catch(e){vArray[vArray.length]=encodeURIComponent("plugin="+e)}break}}vArray=vAr 
ray.join();return vArray}var o={navigator:"exists","navigator.vendor":"value",op 
era:"exists",ActiveXObject:"exists","navigator.appName":"value",platform:"plugin 
",webkitURL:"exists","navigator.plugins.length==0":"value"};try{setIncapCookie(t 
est(o));document.createElement("img").src="/_Incapsula_Resource?SWKMTFSR=1&e="+M 
ath.random()}catch(e){img=document.createElement("img");img.src="/_Incapsula_Res 
ource?SWKMTFSR=1&e="+e}})(); 
</script> 
<script> 
(function() { 
var z="";var b="7472797B766172207868723B76617220743D6E6577204461746528292E676574 
54696D6528293B766172207374617475733D227374617274223B7661722074696D696E673D6E6577 
2041727261792833293B77696E646F772E6F6E756E6C6F61643D66756E6374696F6E28297B74696D 
696E675B325D3D22723A222B286E6577204461746528292E67657454696D6528292D74293B646F63 
756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073 
756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465 
555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B22 
2922297D3B69662877696E646F772E584D4C4874747052657175657374297B7868723D6E65772058 
4D4C48747470526571756573747D656C73657B7868723D6E657720416374697665584F626A656374 
28224D6963726F736F66742E584D4C4854545022297D7868722E6F6E726561647973746174656368 
616E67653D66756E6374696F6E28297B737769746368287868722E72656164795374617465297B63 
61736520303A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072 
657175657374206E6F7420696E697469616C697A656420223B627265616B3B6361736520313A7374 
617475733D6E6577204461746528292E67657454696D6528292D742B223A2073657276657220636F 
6E6E656374696F6E2065737461626C6973686564223B627265616B3B6361736520323A7374617475 
733D6E6577204461746528292E67657454696D6528292D742B223A20726571756573742072656365 
69766564223B627265616B3B6361736520333A7374617475733D6E6577204461746528292E676574 
54696D6528292D742B223A2070726F63657373696E672072657175657374223B627265616B3B6361 
736520343A7374617475733D22636F6D706C657465223B74696D696E675B315D3D22633A222B286E 
6577204461746528292E67657454696D6528292D74293B6966287868722E7374617475733D3D3230 
30297B706172656E742E6C6F636174696F6E2E72656C6F616428297D627265616B7D7D3B74696D69 
6E675B305D3D22733A222B286E6577204461746528292E67657454696D6528292D74293B7868722E 
6F70656E2822474554222C222F5F496E63617073756C615F5265736F757263653F535748414E4544 
4C3D343436383430363736353433323532383538322C313135353632373036373830323435333434 
33392C31373934323137333630323239353234373434352C333830353834222C66616C7365293B78 
68722E73656E64286E756C6C297D63617463682863297B7374617475732B3D6E6577204461746528 
292E67657454696D6528292D742B2220696E6361705F6578633A20222B633B646F63756D656E742E 
637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265 
736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D 
706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B"; 
for (var i=0;i<b.length;i+=2){z=z+parseInt(b.substring(i, i+2), 16)+",";}z = z.s 
ubstring(0,z.length-1); eval(eval('String.fromCharCode('+z+')'));})(); 
</script></head> 
<body> 
<iframe style="display:none;visibility:hidden;" src="//content.incapsula.com/jsT 
est.html" id="gaIframe"></iframe> 
</body> 
</html>

什麼我做錯了任何想法？

來源

2014-07-13 novacourtois

你看到的是一個旨在阻礙程序訪問網站的Incapsula挑戰。除了提交表單字段之外，您還需要滿足通常作爲cookie值提交的Incapsula字段。您的問題不能直接作爲結果回答。 – shux

的問題是以下行：

response = urllib.request.urlopen(request)

你提交申請，要求爲GET而不是POST，導致其失敗。

從docs：

urllib.request.urlopen(url[, data][, timeout])

打開URL鏈接，其可以是字符串或請求對象。

數據可能是指定要發送給服務器的附加數據的字符串，如果不需要此類數據，則爲無。目前HTTP請求是唯一使用數據的請求; 當提供數據參數時，HTTP請求將成爲POST而不是GET。數據應該是標準應用程序/ x-www-form-urlencoded格式的緩衝區。 urllib.parse.urlencode（）函數採用2元組的映射或序列，並以此格式返回字符串。

來源

2014-07-14 00:11:49 alfasin

使用urllib登錄到網站Python 3.x

回答

相關問題