2014-06-25 73 views
0

我開始在紗線集羣上測試oozie作業「oozie job -config /usr/lib/oozie/oozie-4.0.1/examples/apps/pig/job.properties -run」和我的工作停留在0%,然後只發送心臟跳動Oozie示例運行豬作業時卡住

2014-06-25 12:51:57,800 [JobControl] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at localhost/127.0.0.1:8032 
2014-06-25 12:51:57,831 [JobControl] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS 
2014-06-25 12:51:58,021 [JobControl] INFO org.apache.hadoop.mapreduce.lib.input.FileInputFormat - Total input paths to process : 1 
2014-06-25 12:51:58,021 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 
2014-06-25 12:51:58,021 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths to process : 1 
2014-06-25 12:51:58,022 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 
2014-06-25 12:51:58,022 [JobControl] INFO org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input paths (combined) to process : 1 
2014-06-25 12:51:58,034 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1 
2014-06-25 12:51:58,084 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: job_1403700612904_0002 
2014-06-25 12:51:58,084 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Kind: mapreduce.job, Service: job_1403700612904_0001, Ident: ([email protected]ad6884) 
2014-06-25 12:51:58,085 [JobControl] INFO org.apache.hadoop.mapreduce.JobSubmitter - Kind: RM_DELEGATION_TOKEN, Service: 127.0.0.1:8032, Ident: (owner=test, renewer=oozie mr token, realUser=root, issueDate=1403700700415, maxDate=14043$ 
2014-06-25 12:51:58,352 [JobControl] INFO org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted application application_1403700612904_0002 to ResourceManager at localhost/127.0.0.1:8032 
2014-06-25 12:51:58,393 [JobControl] INFO org.apache.hadoop.mapreduce.Job - The url to track the job: http://localhost:8088/proxy/application_1403700612904_0002/ 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1403700612904_0002 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId: job_1403700612904_0002 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases A,B 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Processing aliases A,B 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: A[18,4],B[19,4] C: R: 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - detailed locations: M: A[18,4],B[19,4] C: R: 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002 
2014-06-25 12:51:58,394 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002 
2014-06-25 12:51:58,429 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
2014-06-25 12:51:58,429 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - 0% complete 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
Heart beat 
+0

你能分享你的豬嗎? – Mzf

+0

你可以分享你的workflow.xml和pigscript –

回答

0

而用Hadoop-2.2.0調度豬0.12.1在Oozie的-4.0.1我也面臨着同樣的問題。

我無法在單節點集羣中使用hadoop-2.2.0中的oozie安排豬腳本。但是我通過做了以下更改,在多節點羣集中完成了它。

NodeManager和資源管理器運行在同一系統中。所以我得到這個錯誤。我在第二個系統中啓動了節點管理器,並解決了我的問題。

+0

如果我同時運行兩個以上的oozie工作,意味着它運行失敗,同樣的問題再次提高。因爲運行oozie工作意味着發射器工作被創建時,它佔用一個節點管理器。這種解決方案在所有情況下都沒有幫助。 –

0

Heart beat發生錯誤的原因是Hadoop中的內存總量不足。這是因爲你可能在小型機器上運行。

解決方案: 所以,你想增加總內存的大小,讓所有的nodemanager執行mapreduce作業。 此鏈接的步驟如下"HEART BEAT ERROR SOLUTION"