超慢HttpWebRequest

Question

我正在爲我正在做的項目製作一個網站刮板。我已經使所有工作都很好，但是，加載實際頁面需要F-O-R-E-V-E-R。你可以看到頁面它加載在這裏： MCServerList.Net

這裏是我使用的代碼：

private CONST REQUESTURL as string = "http://www.MCServerList.net/?page=" 
private chunkId as int32 = 1 

Dim req As HttpWebRequest = WebRequest.Create(REQUESTURL & chunkId) 
Dim res As HttpWebResponse = req.GetResponse() 
Dim Stream As Stream = res.GetResponseStream() 

然後我用「流」，並通過HTMLAgilityPack加載它發現免費在線。它快速加載頁面，但是，初始請求通常需要大約20-30秒。

任何幫助，將不勝感激！

Answer 1

我剛剛運行了下面的代碼，並忽略了第一次初始編譯，我平均約爲3.3秒，對於GetResponse()約爲3.3秒，對於Load()約爲0.2秒。你在快速連接？你確定這是瓶頸嗎？

Option Explicit On 
Option Strict On 

Imports System.Net 

Public Class Form1 

    Private Const REQUESTURL As String = "http://www.MCServerList.net/?page=" 
    Private chunkId As Int32 = 1 

    Private Sub Form1_Load(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles MyBase.Load 
     Dim ST As New System.Diagnostics.Stopwatch() 
     ST.Start() 
     Dim req = WebRequest.Create(REQUESTURL & chunkId) 

     Dim res = req.GetResponse() 
     Trace.WriteLine(String.Format("GetResponse() : {0}", ST.Elapsed)) 
     Using Stream As System.IO.Stream = res.GetResponseStream() 
      Dim X As New HtmlAgilityPack.HtmlDocument() 
      X.Load(Stream) 
     End Using 
     Trace.WriteLine(String.Format("Load() : {0}", ST.Elapsed)) 
     ST.Stop() 
    End Sub 
End Class