使用WebRequest登錄網站時發生遠程服務器錯誤

Question

因此，我目前正在嘗試使用WebRequest登錄我的帳戶。 我一直在閱讀有關它，我覺得我想用一個例子來通過試驗和錯誤學習。

這是我使用 Login to website, via C#

的例子所以，當我試着執行我的代碼，它返回一個未處理的異常和這一個

System.Net.WebException: 'The remote server returned an error: (404) Not Found.'

我嘗試逐步執行代碼，我認爲它可能是它試圖在無法發佈的地方發佈信息。 我想在解決此問題之前先獲得確認已成功登錄。 爲了解決此問題，我將用戶名和密碼更改爲虛擬文本。

我在這裏做錯了什麼，什麼是解決這個問題最合乎邏輯的方法？ 在此先感謝。

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12; 

string formUrl = "https://secure.runescape.com/m=weblogin/login.ws"; // NOTE: This is the URL the form POSTs to, not the URL of the form (you can find this in the "action" attribute of the HTML's form tag 
string formParams = string.Format("login-username={0}&login-password={1}", "myUsername", "password"); 
string cookieHeader; 
WebRequest req = WebRequest.Create(formUrl); 
req.ContentType = "application/x-www-form-urlencoded"; 
req.Method = "POST"; 
byte[] bytes = Encoding.ASCII.GetBytes(formParams); 
req.ContentLength = bytes.Length; 
using (Stream os = req.GetRequestStream()) 
{ 
    os.Write(bytes, 0, bytes.Length); 
} 
WebResponse resp = req.GetResponse(); 

cookieHeader = resp.Headers["Set-cookie"]; 

Answer 1

當你抓一個網站，你必須確保你模仿發生的一切。這包括在表單被POST-ed之前發送的任何客戶端狀態（Cookies）。由於大多數網站不喜歡被機器人抓取或操縱，所以它們通常對有效載荷很挑剔。您嘗試控制的網站也是如此。

你已經錯過了

三個重要的事情：

• 你沒有一個初始GET開始讓你有一個CookieContainer所需的cookie。
• 在帖子中，您錯過了表單中的標題（Referrer）和三個隱藏字段。
• 表單域名爲用戶名和密碼（可以在輸入標籤的name屬性中看到）。你已經使用了ID。

修復這些遺漏將導致下面的代碼：

ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12; 
string useragent = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36"; 

// capture cookies, this is important! 
var cookies = new CookieContainer(); 

// do a GET first, so you have the initial cookies neeeded 
string loginUrl = "https://secure.runescape.com/m=weblogin/loginform.ws?mod=www&ssl=0&dest=community"; 
// HttpWebRequest 
var reqLogin = (HttpWebRequest) WebRequest.Create(loginUrl); 
// minimal needed settings 
reqLogin.UserAgent = useragent; 
reqLogin.CookieContainer = cookies; 

reqLogin.Method = "GET"; 
var loginResp = reqLogin.GetResponse(); 
//loginResp.Dump(); // LinqPad testing 

string formUrl = "https://secure.runescape.com/m=weblogin/login.ws"; // NOTE: This is the URL the form POSTs to, not the URL of the form (you can find this in the "action" attribute of the HTML's form tag 
// in ther html the form has 3 more hidden fields, those are needed as well 
string formParams = string.Format("username={0}&password={1}&mod=www&ssl=0&dest=community", "myUsername", "password"); 
string cookieHeader; 
// notice the cast to HttpWebRequest 
var req = (HttpWebRequest) WebRequest.Create(formUrl); 

// put the earlier cookies back on the request 
req.CookieContainer = cookies; 

// the Referrer is mandatory, without it a timeout is raised 
req.Headers["Referrer"] = "https://secure.runescape.com/m=weblogin/loginform.ws?mod=www&ssl=0&dest=community"; 
req.UserAgent = useragent; 

req.ContentType = "application/x-www-form-urlencoded"; 
req.Method = "POST"; 
byte[] bytes = Encoding.ASCII.GetBytes(formParams); 
req.ContentLength = bytes.Length; 
using (Stream os = req.GetRequestStream()) 
{ 
    os.Write(bytes, 0, bytes.Length); 
} 
WebResponse resp = req.GetResponse(); 

cookieHeader = resp.Headers["Set-cookie"]; 

這將返回我的成功。這取決於您解析生成的HTML以計劃您的下一步。