如何從一個網站使用Java複製html div的內容

Question

我想在java中編寫一個函數，該函數基本上將從一個url的div中複製和粘貼html代碼。有問題的數據來自http://cdn.espn.com/sports/scores#completed但是當使用io流複製到我的函數中時，數據不可見。數據本身是可見的，當我點擊檢查和控制 - 「完成足球」它顯示爲，但我的代碼根本不檢索它。這是我使用的代碼。

package project; 

import java.io.BufferedReader; 
import java.io.IOException; 
import java.io.InputStream; 
import java.io.InputStreamReader; 
import java.net.URL; 
import java.net.URLConnection; 


public class DownloadPage { 

    public static void main(String[] args) throws IOException { 

     // Make a URL to the web page 
     URL url = new URL("http://cdn.espn.com/sports/scores#completed-soccer"); 

     // Get the input stream through URL Connection 
     URLConnection con = url.openConnection(); 
     InputStream is =con.getInputStream(); 


     BufferedReader br = new BufferedReader(new InputStreamReader(is)); 

     String line = null; 

     // read each line and write to System.out 
     while ((line = br.readLine()) != null) { 
      System.out.println(line); 
     } 
} 

Answer 1

如果不能達到正常的HTTP請求數據，你必須有一個webdriver的使用更復雜的庫，硒。

這個庫允許你真的在網頁中導航，執行javascript並檢查所有的元素。

你可以找到很多信息和指南。

Answer 2

嘗試使用此代碼

public static void main(String[] args) throws IOException { 
     URL url = new URL("http://cdn.espn.com/sports/scores#completed-soccer"); 
     HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection(); 
     try 
     { 
      InputStream in = url.openStream(); 
      BufferedReader reader = new BufferedReader(new InputStreamReader(in)); 
      StringBuilder result = new StringBuilder(); 
      String line; 
      while((line = reader.readLine()) != null) { 
       result.append(line); 
      } 
      System.out.println(result.toString()); 
     } 
     finally 
     { 
      urlConnection.disconnect(); 
     } 
    }