設置數據節點時Hadoop中的特權操作異常？

Question

我在我的Mac OS X中設置了hadoop框架。使用virtualbox，我安裝了ubuntu服務器（一個nameNode）和兩個更多的ubuntu服務器來充當datanode。 我已經正確設置了core-site.xml和其他配置，據我所知。當我使用start-all.sh啓動框架時，一切似乎都很好。在NameNode上執行jps，每個datanode顯示進程處於活動狀態。但在GUI網址上，我看到沒有節點是活的。看到附加的圖像。 下面是從日誌文件中的錯誤消息：

2014-06-23 18:00:01,167 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: Web-server up at: 0.0.0.0:50070 
2014-06-23 18:00:01,174 INFO org.apache.hadoop.ipc.Server: IPC Server Responder: starting 
2014-06-23 18:00:01,175 INFO org.apache.hadoop.ipc.Server: IPC Server listener on 10001: starting 
2014-06-23 18:00:01,178 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001: starting 
2014-06-23 18:00:01,186 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on 10001: starting 
2014-06-23 18:00:01,190 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 10001: starting 
2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on 10001: starting 
2014-06-23 18:00:01,195 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001: starting 
2014-06-23 18:00:01,196 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on 10001: starting 
2014-06-23 18:00:01,198 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on 10001: starting 
2014-06-23 18:00:01,201 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on 10001: starting 
2014-06-23 18:00:01,206 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on 10001: starting 
2014-06-23 18:00:01,213 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on 10001: starting 
2014-06-23 18:00:05,692 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 
2014-06-23 18:00:05,692 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 
java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 
     at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1920) 
     at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:783) 
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
     at java.lang.reflect.Method.invoke(Method.java:606) 
     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587) 
     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432) 
     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190) 
     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426) 
2014-06-23 18:00:06,716 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:sridhar cause:java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 
2014-06-23 18:00:06,716 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on 10001, call addBlock(/usr/local/hadoop/tmp/mapred/system/jobtracker.info, DFSClient_NONMAPREDUCE_1288778011_1, null) from 127.0.0.1:56524: error: java.io.IOException: File /usr/local/hadoop/tmp/mapred/system/jobtracker.info could only be replicated to 0 nodes, instead of 1 

PS：我沒有尋找其他的職位，顯示出類似的錯誤。我試過了。但沒有成功

編輯：

名稱節點服務器不顯示在發行jps命令數據節點和的TaskTracker。 的數據節點的日誌如下所示：（可能的DataNode沒有建立連接）

2014-06-23 17:14:16,003 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:17,005 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS) 2014-06-23 17:14:18,006 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: ubuntuhadoop/69.91.135.178:10001. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)

Answer 1

我得到了這個工作。雖然解決方案很簡單，但我想在此發佈，以便其他新手Hadoopers可能會受益。

1）具有core-site.xml，hdfs-site.xml和mapred-site.xml的精確的複製品在兩個主（nameNode）和所有從站（dataNodes）。我認爲core-site.xml和mapred-site.xml在主人並不重要。但它是。他們打開他們聽的港口。在這些端口上，dataNode可以達到nameNode。

2）當你在主機上運行jps，你不會看到一個過程（至少我沒看到）爲DataNodes和TaskTracker假設你從主做start-all.sh。原因是因爲DataNodes和TaskTracker線程是在集羣的不同節點上處理的（在我的情況下，我有一個作爲master的ubuntu盒子和兩個作爲datanode的ubuntu盒子）。

注意：在我的情況下，我沒有讓我的主人，作爲數據管理部（即作爲從機）

Answer 2

它從你的主節點無法連接數據節點之一（ubuntuhadoop/69.91日誌似乎.135.178：10001）。

作爲初始測試，請ssh從主數據節點，看看你是否可以連接它。 以下鏈接可能有助於設置羣集！

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/