h2o從YARN開始不起作用

Question

當我在cdh集羣上啓動H2o時，出現以下錯誤。我下載了他所有的東西，然後按照教程。我跑的命令是

hadoop jar h2odriver.jar -nodes 2 -mapperXmx 1g -output hdfsOutputDirName 

它顯示容器沒有被使用。目前還不清楚這些會在hadoop上設置什麼。我給了所有的設置內存。這是內存的0.0，沒有意義，爲什麼容器不使用內存。羣集現在是否正在運行？

----- YARN cluster metrics ----- 
Number of YARN worker nodes: 3 

----- Nodes ----- 
Node: http://data-node-3:8042 Rack: /default, RUNNING, 1 containers used, 1.0/6.0 GB used, 1/4 vcores used 
Node: http://data-node-1:8042 Rack: /default, RUNNING, 0 containers used, 0.0/6.0 GB used, 0/4 vcores used 
Node: http://data-node-2:8042 Rack: /default, RUNNING, 0 containers used, 0.0/6.0 GB used, 0/4 vcores used 

----- Queues ----- 
Queue name:   root.default 
    Queue state:  RUNNING 
    Current capacity: 0.00 
    Capacity:   0.00 
    Maximum capacity: -1.00 
    Application count: 0 

Queue 'root.default' approximate utilization: 0.0/0.0 GB used, 0/0 vcores used 

---------------------------------------------------------------------- 

WARNING: Job memory request (2.2 GB) exceeds queue available memory capacity (0.0 GB) 
WARNING: Job virtual cores request (2) exceeds queue available virtual cores capacity (0) 

---------------------------------------------------------------------- 

For YARN users, logs command is 'yarn logs -applicationId application_1462681033282_0008' 

Answer 1

我做Cloudera Manager紗線配置中的以下變化

Setting          Value 
yarn.scheduler.maximum-allocation-vcores 8 
yarn.nodemanager.resource.cpu-vcores  4 
yarn.nodemanager.resource.cpu-vcores  4 
yarn.scheduler.maximum-allocation-mb  16 GB 

Answer 2

您應該設置默認隊列以擁有運行2節點集羣的可用資源。

見警告：

1. WARNING: Job memory request (2.2 GB) exceeds queue available memory capacity (0.0 GB)

   • 你問每個節點1GB（+開銷），但沒有可用的資源在紗線隊列

2. WARNING: Job virtual cores request (2) exceeds queue available virtual cores capacity (0)

   • 你問2個虛擬內核，但沒有核心的默認隊列可

請檢查YARN文件 - 容量調度的示例設置，最大可利用的資源： https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html