我試圖通過Hadoop MapReduce程序對Elasticsearch進行查詢,但無法運行它的jar。下面是錯誤:無法從Hadoop MapReduce程序在Elasticsearch上搜索
Exception in thread "main" java.lang.NoClassDefFoundError: org/elasticsearch/common/transport/TransportAddress
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.util.RunJar.main(RunJar.java:153)
Caused by: java.lang.ClassNotFoundException: org.elasticsearch.common.transport.TransportAddress
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 3 more
此前我已加載其他庫,例如共配置,共郎但我沒有遇到問題的時間。應該做什麼改變才能正確工作。
我已經嘗試構建正常的應用程序並運行它進行elasticsearch。
在Hadoop的lib目錄中添加所有必需的庫之後,我現在能夠從主函數上搜索Elasticsearch數據庫,但是當我嘗試從map函數執行相同的搜索時,出現以下錯誤。
16/01/07 18:44:44 INFO mapred.JobClient: Task Id : attempt_201601011215_30692_m_000000_0, Status : FAILED
Error: java.lang.ClassNotFoundException: org.elasticsearch.client.Client
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at fragmentextractor.FragmentExtractor$FragmentExtractorMapper.is_clean(FragmentExtractor.java:163)
at fragmentextractor.FragmentExtractor$FragmentExtractorMapper.writeToContext(FragmentExtractor.java:216)
at fragmentextractor.FragmentExtractor$FragmentExtractorMapper.map(FragmentExtractor.java:282)
at fragmentextractor.FragmentExtractor$FragmentExtractorMapper.map(FragmentExtractor.java:68)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
據我所知,它能夠在父進程中搜索和加載庫,但是當它克隆時,圖書館不能在兒童中找到。還有哪些地方可以使庫文件保持成功?
映射器中缺少相應的jar。將正確的jar添加到路徑中。 – YoungHobbit
我從NetBeans IDE創建了jar,併爲其添加了所有必需的jar文件。我也檢查了在PATH環境變量中添加jar但沒有成功。 –
只需提取您的NetBeans創建的jar並檢查它是否具有相應的jar? – YoungHobbit