使用Capacity Scheduler在Hadoop集羣中設置隊列

Question

我嘗試設置3個隊列。爲此我配置了CapacityScheduler.xml，如下所示。但在這之後，求職者正在關閉。我還將這些隊列的名稱添加到了mapred-site.xml中。任何人都知道原因。

enter code here 

*<!-- queue: BatchJobs-Queue --> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.capacity</name> 
    <value>70</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.supports-priority</name> 
    <value>false</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.minimum-user-limit-percent</name> 
    <value>20</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.user-limit-factor</name> 
    <value>1</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.maximum-initialized-active-tasks</name> 
    <value>200000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.maximum-initialized-active-tasks-per-user</name> 
    <value>100000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.BatchJobsQueue.init-accept-jobs-factor</name> 
    <value>10</value> 
</property> 

    enter code here 

<!-- queue: AdHocJobs-Queue --> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.capacity</name> 
    <value>20<value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.supports-priority</name> 
    <value>false</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.minimum-user-limit-percent</name> 
    <value>20</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.user-limit-factor</name> 
    <value>1</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.maximum-initialized-active-tasks</name> 
    <value>200000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.maximum-initialized-active-tasks-per-user</name> 
    <value>100000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.AdHocJobsQueue.init-accept-jobs-factor</name> 
    <value>10</value> 
</property> 

    enter code here 

<!-- queue: MyQueue --> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.capacity</name> 
    <value>10<value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.supports-priority</name> 
    <value>false</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.minimum-user-limit-percent</name> 
    <value>20</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.user-limit-factor</name> 
    <value>1</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.maximum-initialized-active-tasks</name> 
    <value>200000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.maximum-initialized-active-tasks-per-user</name> 
    <value>100000</value> 
</property> 
<property> 
    <name>mapred.capacity-scheduler.queue.MyQueue.init-accept-jobs-factor</name> 
    <value>10</value> 
</property>* 

enter code here 

Answer 1

您的作業跟蹤器日誌的輸出是什麼？

假設你正在運行Linux嘗試運行以下命令：

ls -lrt /var/log/hadoop/mapred/ | grep jobtracker | tail -n1 | awk '{print $9}' | xargs -I{} tail -n40 /var/log/hadoop/mapred/{}