2014-07-02 90 views
0

我想在兩臺機器上運行hadoop。下面提到的是我的core-site.xml文件。Hadoop集羣設置:數據節點不能在奴隸機上工作

10.0.x.y作爲我的主機工作,10.0.x.y1作爲從機工作。

<configuration> 
    <property> 
     <name>fs.default.name</name> 
     <value>hdfs://10.0.x.y:54310</value> 
    </property> 
</configuration> 

下文提到的是我的HDFS-site.xml文件

<configuration> 
     <property> 
     <name>dfs.replication</name> 
     <value>1</value> 
    </property> 
    <property> 
     <name>dfs.name.dir</name> 
     <value>/data/hadoop-namenode-data</value> 
    </property> 
    <property> 
     <name>dfs.data.dir</name> 
     <value>/data/hadoop-datanode-data</value> 
    </property> 

</configuration> 

有關10.0.xy start-dfs.sh命令---我的名字節點和數據節點的機器上10.0。 xy運行良好,但namenode.log文件上的錯誤消息是

2014-07-02 15:49:08,628 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: NameNode RPC up at: TPMOPW388/10.0.7.111:54310 
2014-07-02 15:49:08,628 INFO org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Starting services required for active state 
2014-07-02 15:49:11,268 WARN org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager: Unresolved datanode registration from 10.0.40.18 
2014-07-02 15:49:11,268 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0) 
2014-07-02 15:49:11,268 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on 54310, call org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from 10.0.40.18:57523 Call#1 Retry#0: error: org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0) 
org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0) 
     at org.apache.hadoop.hdfs.server.blockmanagement.DatanodeManager.registerDatanode(DatanodeManager.java:739) 
     at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.registerDatanode(FSNamesystem.java:3929) 
     at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.registerDatanode(NameNodeRpcServer.java:948) 
     at org.apache.hadoop.hdfs.protocolPB.DatanodeProtocolServerSideTranslatorPB.registerDatanode(DatanodeProtocolServerSideTranslatorPB.java:90) 
     at org.apache.hadoop.hdfs.protocol.proto.DatanodeProtocolProtos$DatanodeProtocolService$2.callBlockingMethod(DatanodeProtocolProtos.java:24079) 
     at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) 
     at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928) 
     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048) 
     at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044) 
     at java.security.AccessController.doPrivileged(Native Method) 
     at javax.security.auth.Subject.doAs(Subject.java:415) 
     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491) 
     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042) 

任何幫助表示讚賞。

回答

1

可能與主機名配置有關。檢查/etc/hosts文件,並確保您有每臺機器的IP地址和主機名。

0

在hdfs-site.xml 中使用值=「系統中的節點數」,並檢查/ etc/hosts。

0

根據這條線:

15:49:11,268 ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:root (auth:SIMPLE) cause:org.apache.hadoop.hdfs.server.protocol.DisallowedDatanodeException: Datanode denied communication with namenode: DatanodeRegistration(0.0.0.0, storageID=DS-969520512-10.0.40.18-50010-1404290045109, infoPort=50075, ipcPort=50020, storageInfo=lv=-47;cid=CID-9afd9260-85c3-4018-92a6-cb391f4cfbdd;nsid=471345421;c=0) 

這可能是由你的名字節點導致無法通過ssh以root戶頭訪問您的數據節點。

所以,

確保如果安裝ssh和配置私有密鑰,從而無需密碼登錄!