無需使用bin/nutch從java web應用程序抓取網站

我想從我的（java）mojarra 2.0.2 webapp使用nutch（1.1）而不使用bin/nutch ...我在google搜索示例，但有沒有例子我怎麼能認識到這一點：/ ...我得到一個異常和作業失敗：/（我想原因的東西用Hadoop）...這裏是我的代碼：無需使用bin/nutch從java web應用程序抓取網站

 
    public void run() throws Exception { 
     final String[] args = new String[] { 
      String.format("%s%s%s%s", JSFUtils.getWebAppRoot(), "nutch", File.separator, DIRECTORY_URLS), 
      "-dir", String.format("%s%s%s%s", JSFUtils.getWebAppRoot(), "nutch", File.separator, DIRECTORY_CRAWL), 
      "-threads", this.preferences.get("threads"), 
      "-depth", this.preferences.get("depth"), 
      "-topN", this.preferences.get("topN"), 
      "-solr", this.preferences.get("solr") 
     }; 
     Crawl.main(args); 
    }

和部分日誌記錄：

 
10/05/17 10:42:54 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId= 
10/05/17 10:42:54 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same. 
10/05/17 10:42:54 INFO mapred.FileInputFormat: Total input paths to process : 1 
10/05/17 10:42:54 INFO mapred.JobClient: Running job: job_local_0001 
10/05/17 10:42:54 INFO mapred.FileInputFormat: Total input paths to process : 1 
10/05/17 10:42:55 INFO mapred.MapTask: numReduceTasks: 1 
10/05/17 10:42:55 INFO mapred.MapTask: io.sort.mb = 100 
java.io.IOException: Job failed! 
     at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1232) 
     at org.apache.nutch.crawl.Injector.inject(Injector.java:211) 
     at org.apache.nutch.crawl.Crawl.main(Crawl.java:124) 
     at lan.localhost.process.NutchCrawling.run(NutchCrawling.java:108) 
     at lan.localhost.main.Index.indexing(Index.java:71) 
     at lan.localhost.bean.FeedingBean.actionStart(FeedingBean.java:25) 
     ....

有人可以幫助我或告訴我如何從Java應用程序爬行嗎？我增加了X毫秒爲256M和XMX到768米，但什麼都沒有改變......

問候馬塞爾

來源

2010-05-17 Marcel