使用Web客戶端加載HTML

Question

我試圖下載網頁：

string remoteUri = "http://whois.domaintools.com/94.100.179.159"; 
WebClient myWebClient = new WebClient(); 
byte[] myDataBuffer = myWebClient.DownloadData(remoteUri); 
string download = Encoding.ASCII.GetString(myDataBuffer); 
HtmlDocument doc = new HtmlDocument(); 
doc.LoadHtml(download); 
doc.Save("file1.htm"); 

有錯誤

引發WebException是未處理：禁止（403）。

有沒有其他方法可以下載頁面？ 我試過HtmlDocument類，但正如我所見，它需要在瀏覽器中加載網頁。

HtmlWeb hwObject = new HtmlWeb(); 
     string ip = "http://whois.domaintools.com/"; 
     HtmlAgilityPack.HtmlDocument htmldocObject = hwObject.Load(ip); 

     foreach (HtmlNode link in htmldocObject.DocumentNode.SelectNodes("//meta[@name = 'description']")) 
     { 
      ... 
     } 

Answer 1

using (var myWebClient = new WebClient()) 
{ 
    myWebClient.Headers["User-Agent"] = "MOZILLA/5.0 (WINDOWS NT 6.1; WOW64) APPLEWEBKIT/537.1 (KHTML, LIKE GECKO) CHROME/21.0.1180.75 SAFARI/537.1"; 

    string page = myWebClient.DownloadString("http://whois.domaintools.com/94.100.179.159"); 

    HtmlDocument doc = new HtmlDocument(); 
    doc.LoadHtml(page); 
} 

Answer 2

該網站只是在請求中找不到用戶代理時返回錯誤，下面是工作代碼。

string remoteUri = "http://whois.domaintools.com/94.100.179.159"; 
HtmlDocument doc = new HtmlDocument(); 
using (WebClient myWebClient = new WebClient()) 
{ 
    myWebClient.Headers.Add(HttpRequestHeader.UserAgent, "some browser user agent"); 
    doc.Load(myWebClient.OpenRead(remoteUri)); 
} 
doc.Save("file1.htm"); 

或者，如果你想使用HtmlWeb

HtmlWeb hwObject = new HtmlWeb(); 
hwObject.UserAgent = "some browser user agent"; 
//more code...