使用Crawler4j線程「主」java.lang.NoClassDefFoundError異常：org/apache/http/conn/scheme/SchemeSocketFactory

Question

我正在使用Crawler4j示例代碼，但我發現我有一個異常。

這裏是我的例外：

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/http/conn/scheme/SchemeSocketFactory 
    at LocalDataCollectorController.main(LocalDataCollectorController.java:24) 
Caused by: java.lang.ClassNotFoundException: org.apache.http.conn.scheme.SchemeSocketFactory 

這裏是我的代碼：

public static void main(String[] args) throws Exception { 

      String root Folder = "D:\\"; 
      int numberOfCrawlers = 5; 
      System.out.println("numberOfCrawlers"+numberOfCrawlers); 
      System.out.println(rootFolder); 

      CrawlConfig config = new CrawlConfig(); 
      config.setCrawlStorageFolder(rootFolder); 
      config.setMaxPagesToFetch(10); 
      config.setPolitenessDelay(1000); 

       PageFetcher pageFetcher = new PageFetcher(config); 
      RobotstxtConfig robotstxtConfig = new RobotstxtConfig(); 
      RobotstxtServer robotstxtServer = new RobotstxtServer(robotstxtConfig, pageFetcher); 
      CrawlController controller = new CrawlController(config, pageFetcher, robotstxtServer); 

      controller.addSeed("http://www.ohloh.net/p/crawler4j"); 
      controller.start(LocalDataCollectorCrawler.class, numberOfCrawlers); 

      List<Object> crawlersLocalData = controller.getCrawlersLocalData(); 
      long totalLinks = 0; 
      long totalTextSize = 0; 
      int totalProcessedPages = 0; 
      for (Object localData : crawlersLocalData) { 
        CrawlStat stat = (CrawlStat) localData; 
        totalLinks += stat.getTotalLinks(); 
        totalTextSize += stat.getTotalTextSize(); 
        totalProcessedPages += stat.getTotalProcessedPages(); 
      } 
      System.out.println("Aggregated Statistics:"); 
      System.out.println(" Processed Pages: " + totalProcessedPages); 
      System.out.println(" Total Links found: " + totalLinks); 
      System.out.println(" Total Text Size: " + totalTextSize); 
    } 

} 

Answer 1

原因NoClassDefFoundError始終是相同的：你運行時執行過程中沒有提供相關性類。換句話說，當你運行你的例子時，你並沒有將HttpClient的JAR文件放在類路徑中。做到這一點，問題就會消失。

Answer 2

下載HttpClient並將其添加到您的構建路徑。

還有一個包含download section中所有crawler4j依賴關係的包。你應該用這個來避免更多的問題。