2014-12-02 26 views
0

當我在蜂巢命令行中運行:蜂巢工作出現的MapReduce錯誤:致電HMASTER/127.0.0.1到本地主機:44849失敗的連接異常

hive > select count(*) from alogs; 

在終端,它顯示以下內容:

Total jobs = 1 
Launching Job 1 out of 1 
Number of reduce tasks determined at compile time: 1 
In order to change the average load for a reducer (in bytes): 
    set hive.exec.reducers.bytes.per.reducer=<number> 
In order to limit the maximum number of reducers: 
    set hive.exec.reducers.max=<number> 
In order to set a constant number of reducers: 
    set mapreduce.job.reduces=<number> 
Starting Job = job_1417084377943_0009, Tracking URL = http://localhost:8088/proxy/application_1417084377943_0009/ 
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1417084377943_0009 
Hadoop job information for Stage-1: number of mappers: 0; number of reducers: 0 
2014-12-02 17:59:44,068 Stage-1 map = 0%, reduce = 0% 
Ended Job = job_1417084377943_0009 with errors 
Error during job, obtaining debugging information... 
**FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask** 
MapReduce Jobs Launched: 
Stage-Stage-1: HDFS Read: 0 HDFS Write: 0 FAIL 
Total MapReduce CPU Time Spent: 0 msec 

然後我用的ResourceManager看到錯誤的詳細信息:

Application application_1417084377943_0009 failed 2 times due to Error launching appattempt_1417084377943_0009_000002. Got exception: **java.net.ConnectException: Call From hmaster/127.0.0.1 to localhost:44849 failed on connection exception: java.net.ConnectException: Connection refused;** For more details see: http://wiki.apache.org/hadoop/ConnectionRefused 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:408) 
    at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:783) 
    at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:730) 
    at org.apache.hadoop.ipc.Client.call(Client.java:1415) 
    at org.apache.hadoop.ipc.Client.call(Client.java:1364) 
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) 
    at com.sun.proxy.$Proxy32.startContainers(Unknown Source) 
    at org.apache.hadoop.yarn.api.impl.pb.client.ContainerManagementProtocolPBClientImpl.startContainers(ContainerManagementProtocolPBClientImpl.java:96) 
    at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.launch(AMLauncher.java:119) 
    at org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher.run(AMLauncher.java:254) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at java.lang.Thread.run(Thread.java:745) 
    Caused by: java.net.ConnectException: Connection refused 
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) 
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:712) 
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206) 
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529) 
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493) 
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:606) 
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:700) 
    at org.apache.hadoop.ipc.Client$Connection.access$2800(Client.java:367) 
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1463) 
    at org.apache.hadoop.ipc.Client.call(Client.java:1382) 
    ... 9 more 
    . Failing the application. 

雖然錯誤通知是詳細編輯不夠,我不知道在哪裏設置配置'localhost:44849',以及'從hmaster/127.0.0.1到localhost:44849在連接異常時失敗'的含義是什麼?

回答

0

我有同樣的當我運行由春紗編寫的應用程序時遇到問題。 我可能會找到一個解決方案,並多次測試紗線應用,並沒有得到這個錯誤。

首先,修改所有療法服務器的/ etc/hosts和寫所有的奴隸在文件中,就像:

192.168.0.101 slave1 
192.168.0.102 slave2 
... 

其次,修改所有療法服務器的紗的site.xml在/ home /用戶/ Hadoop的在/ etc/Hadoop的/ 並添加屬性,如:

<property> 
    <name>yarn.nodemanager.address</name> 
    <value>slave1:57799</value> 
    </property> 

注意域必須以相同的服務器和端口您可以設置randorm數字,如57799.該端口號必須在一致的所有yarn-site.xml文件。

三,重新啓動resourcemanager和所有nodemanagers。

我希望這可以幫助你。

而且我覺得這個問題我套管,因爲我沒加奴隸列出文件

/home/user/hadoop/etc/hadoop/slaves 

的,但我沒有對此進行測試。

0

如果你的hadoop安裝文件中有一個配置文件「..../hadoop-2.8.1/etc/hadoop/mapred-site.xml」,並且你沒有運行YARN,那麼配置單元任務可能會拋出重試連接到服務器:0.0.0.0/0.0.0.0:8032「異常。 (你可能會發現select *是ok,select sum()是錯誤的,┭┮﹏┭┮)

你可以執行「jps」來檢查YARN是否在運行。

如果紗線沒有運行,結果可能會喜歡:

[[email protected] conf]$ jps 
36721 Jps 
8402 DataNode 
35458 RunJar 
8659 SecondaryNameNode 
8270 NameNode 

如果紗線運行,其結果可能會喜歡:

[[email protected] sbin]$ jps 
13237 Jps 
9767 DataNode 
9975 SecondaryNameNode 
12651 ResourceManager (多了這個) 
12956 NodeManager (多了這個) 
9581 NameNode 
13135 JobHistoryServer 

解決辦法有兩個:

1.rename mapred-site.xml文件,執行linux命令「mv mapred-site.xml mapred-site.xml.template」或刪除mapred-site.xml文件,然後重新啓動hadoop。

2.run YARN。 ps:修改hadoop配置並使用「start-yarn.sh」運行YARN。

相關問題