Hadoop的DataStreamer例外：文件只能被複制到0的節點，而不是minReplication（= 1）

我試圖從我的本地加載JSON數據到hadoop的HDFS，我使用這些命令，並且將其擲異常：Hadoop的DataStreamer例外：文件只能被複制到0的節點，而不是minReplication（= 1）

hadoop fs -copyFromLocal path/files/file.json input/ 
hadoop fs -put path/files/file.json input/

我檢查使用jps命令，發現hadoop正在運行。

26039 ResourceManager 
30858 SecondaryNameNode 
35605 Jps 
26147 NodeManager 
30714 DataNode

這是例外的細節：

WARN hdfs.DFSClient: DataStreamer Exception org.apache.hadoop.ipc.RemoteException(java.io.IOException): File path/files/file.json._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation. 
    at org.apache.hadoop.hdfs.server.blockmanagement.BlockManager.chooseTarget(BlockManager.java:1471) 
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2791) 
    at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:606) 
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:455) 
    at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java) 
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585) 
    at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928) 
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013) 
    at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at javax.security.auth.Subject.doAs(Subject.java:415) 
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614) 
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2007) 

    at org.apache.hadoop.ipc.Client.call(Client.java:1411) 
    at org.apache.hadoop.ipc.Client.call(Client.java:1364) 
    at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206) 
    at com.sun.proxy.$Proxy9.addBlock(Unknown Source) 
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
    at java.lang.reflect.Method.invoke(Method.java:606) 
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187) 
    at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102) 
    at com.sun.proxy.$Proxy9.addBlock(Unknown Source) 
    at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.addBlock(ClientNamenodeProtocolTranslatorPB.java:368) 
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.locateFollowingBlock(DFSOutputStream.java:1449) 
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.nextBlockOutputStream(DFSOutputStream.java:1270) 
    at org.apache.hadoop.hdfs.DFSOutputStream$DataStreamer.run(DFSOutputStream.java:526) 
copyFromLocal: File path/files/file.json._COPYING_ could only be replicated to 0 nodes instead of minReplication (=1). There are 0 datanode(s) running and no node(s) are excluded in this operation.

我應該怎麼辦？我需要將數據加載到hdfs中。在此先感謝。

來源

2015-03-31 nvl

你可以檢查數據節點的日誌。你可能會找到這個問題的原因。 – 2015-03-31 13:36:26

你在哪裏NameNode？啓動它。 – user2018791 2015-03-31 14:07:36

@GauravMishra好的，我會檢查日誌 – nvl 2015-03-31 23:57:42

檢查jps，start-all.sh後需要6個進程。在這裏你開始失敗的NameNode進程。

請格式化的NameNode：

的Hadoop的NameNode -format

來源

2017-09-07 10:23:16

有時我必須在格式化namenode之前刪除datanode和namenode文件夾。 – 2017-10-04 03:11:52

Hadoop的DataStreamer例外：文件只能被複制到0的節點，而不是minReplication（= 1）

回答

相關問題