2016-02-08 70 views
0

我對Hadoop的奇蹟比較陌生,但我正在嘗試使用官方的Hadoop 2.7.2版Apache Hadoop文檔手動構建一個集羣。當我運行命令:$HADOOP_PREFIX/sbin/hadoop-daemon.sh start namenode我返回到我的控制檯並顯示一條消息,指出守護程序正在啓動,並且.out文件的位置。當我vim的該文件路徑,我會見了:Hadoop namenode守護進程在5秒鐘後靜默停止

ulimit -a for user hadoop 
core file size   (blocks, -c) 0 
data seg size   (kbytes, -d) unlimited 
scheduling priority    (-e) 0 
file size    (blocks, -f) unlimited 
pending signals     (-i) 15017 
max locked memory  (kbytes, -l) 64 
max memory size   (kbytes, -m) unlimited 
open files      (-n) 1024 
pipe size   (512 bytes, -p) 8 
POSIX message queues  (bytes, -q) 819200 
real-time priority    (-r) 0 
stack size    (kbytes, -s) 8192 
cpu time    (seconds, -t) unlimited 
max user processes    (-u) 4096 
virtual memory   (kbytes, -v) unlimited 
file locks      (-x) unlimited 

再次運行啓動NameNode的命令會給我守護進程開始了同樣的信息和日誌文件(同一地點)的位置。

這是當我在看.log文件我得到什麼:

2016-02-03 16:03:04,092 INFO org.apache.hadoop.hdfs.server.namenode.NameNode: STARTUP_MSG: 
/************************************************************ 
STARTUP_MSG: Starting NameNode 
STARTUP_MSG: host = namenode_dns_name/127.0.1.1 
STARTUP_MSG: args = [] 
STARTUP_MSG: version = 2.7.2 

我所知道的,到目前爲止是:

  • 在我的配置文件的語法錯誤將導致錯誤在日誌文件的頂部
  • 我的namenode守護進程在啓動後大約5秒鐘無聲無息地崩潰
  • 我的配置文件中的錯誤設置可能導致第ËNameNode會默默地守護崩潰
  • 我沒有,我已經在其他問題
  • 看見提及此JPS命令

我不知道這是我的Java版本的問題,但我已經嘗試使用openjdk-1.8.0.65-3和openjdk-1.7.0.95(這兩者都沒有在官方Hadoop Java版本頁面上列出http://wiki.apache.org/hadoop/HadoopJavaVersions,但我希望我不只限於這些版本)

I也不知道它是否是我的配置文件的問題,但我會將它們包括在這裏以供審查(儘管主機名被阻止)。並可列出調試所需的其他任何信息。另外,我試圖在同一臺機器上運行namenode守護進程和resourcemanager守護進程以進行測試。

謝謝你的時間。

HDFS-site.xml中

<?xml version="1.0" encoding="UTF-8"?> 
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> 
<!-- 
    Licensed under the Apache License, Version 2.0 (the "License"); 
    you may not use this file except in compliance with the License. 
    You may obtain a copy of the License at 

    http://www.apache.org/licenses/LICENSE-2.0 

    Unless required by applicable law or agreed to in writing, software 
    distributed under the License is distributed on an "AS IS" BASIS, 
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
    See the License for the specific language governing permissions and 
    limitations under the License. See accompanying LICENSE file. 
--> 

<!-- Put site-specific property overrides in this file. --> 
<configuration> 
    <property> 
     <name>dfs.namenode.name.dir</name> 
     <value>file:///home/hadoop/hadoop-2.7.2/hdfs/namenode</value> 
     <description>Path on the local filesystem where the NameNode stores the namespace and transaction logs persistently.</description> 
    </property> 
    <property> 
     <name>dfs.hosts</name> 
     <value>datanode_dns_name</value> 
    </property> 
    <property> 
     <name>dfs.blocksize</name> 
     <value>268435456</value> 
    </property> 
    <property> 
     <name>dfs.namenode.handler.count</name> 
     <value>100</value> 
    </property> 
</configuration> 

芯-site.xml中

<?xml version="1.0" encoding="UTF-8"?> 
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?> 
<!-- 
    Licensed under the Apache License, Version 2.0 (the "License"); 
    you may not use this file except in compliance with the License. 
    You may obtain a copy of the License at 

    http://www.apache.org/licenses/LICENSE-2.0 

    Unless required by applicable law or agreed to in writing, software 
    distributed under the License is distributed on an "AS IS" BASIS, 
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
    See the License for the specific language governing permissions and 
    limitations under the License. See accompanying LICENSE file. 
--> 

<!-- Put site-specific property overrides in this file. --> 
<configuration> 
    <property> 
     <name>hadoop.tmp.dir</name> 
     <value>/app/hadoop/tmp</value> 
    </property> 
    <property> 
     <name>fs.defaultFS</name> 
     <value>hdfs://namenode_dns_name</value> 
     <description>NameNode URI</description> 
    </property> 
    <property> 
     <name>io.file.buffer.size</name> 
     <value>131072</value> 
    </property> 
</configuration> 

紗線的site.xml

<?xml version="1.0"?> 
<!-- 
    Licensed under the Apache License, Version 2.0 (the "License"); 
    you may not use this file except in compliance with the License. 
    You may obtain a copy of the License at 

    http://www.apache.org/licenses/LICENSE-2.0 

    Unless required by applicable law or agreed to in writing, software 
    distributed under the License is distributed on an "AS IS" BASIS, 
    WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
    See the License for the specific language governing permissions and 
    limitations under the License. See accompanying LICENSE file. 
--> 
<configuration> 
    <property> 
     <name>yarn.acl.enable</name> 
     <value>false</value> 
    </property> 
    <property> 
     <name>yarn.admin.acl</name> 
     <value> </value> 
    </property> 
    <property> 
     <name>yarn.log-aggregation-enable</name> 
     <value>false</value> 
    </property> 
    <property> 
     <name>yarn.resourcemanager.host</name> 
     <value>namenode_dns_name</value> 
    </property> 
    <property> 
     <name>yarn.scheduler.minimum-allocation-mb</name> 
     <value>128</value> 
     <description>Minimum limit of memory to allocate to each container request at the Resource Manager.</description> 
    </property> 
    <property> 
     <name>yarn.scheduler.maximum-allocation-mb</name> 
     <value>2048</value> 
     <description>Maximum limit of memory to allocate to each container request at the Resource Manager.</description> 
    </property> 
    <property> 
     <name>yarn.scheduler.minimum-allocation-vcores</name> 
     <value>1</value> 
     <description>The minimum allocation for every container request at the RM, in terms of virtual CPU cores. Requests lower than this won't take effect, and the specified value will get allocated the minimum.</description> 
    </property> 
    <property> 
     <name>yarn.scheduler.maximum-allocation-vcores</name> 
     <value>2</value> 
     <description>The maximum allocation for every container request at the RM, in terms of virtual CPU cores. Requests higher than this won't take effect, and will get capped to this value.</description> 
    </property> 
    <property> 
     <name>yarn.nodemanager.resource.memory-mb</name> 
     <value>4096</value> 
     <description>Physical memory, in MB, to be made available to running containers</description> 
    </property> 
    <property> 
     <name>yarn.nodemanager.resource.cpu-vcores</name> 
     <value>4</value> 
     <description>Number of CPU cores that can be allocated for containers.</description> 
    </property> 
</configuration> 
+0

你可以發佈namenode日誌 – BruceWayne

+0

@Krishna在.out文件內容下添加.log文件的內容。 –

+0

添加完整的日誌...否則沒有用 – BruceWayne

回答

0

日誌文件中指示該過程引發了一個非綁定IOException由於所需的端口已被佔用。我現在也可以添加到已知事物的列表中,vim中的「@」表示單個行字符串超出行大小,而不是文件末尾。