Oozie和sqoop配置問題

我想設置一個oozie和sqoop工作流（我想將mySql數據備份到我的hdfs中）。但是當我嘗試啓動我的工作時，我陷入了困境。Oozie和sqoop配置問題

我正在使用hadoop2（工作hdfs節點），oozie的最後一個版本。我在計算機上安裝了oozie服務器（我想在部署它之前對其進行測試），在oozie conf上使用hdfs配置（core-site.xml，hdfs-site.xml，yarn-site.xml，mapred-site.xml）/haddop-conf dir）和我的hdfs在服務器上。

我已經做了基本工作流程（測試目的，我只是想看看是否sqoop工作）這樣的：

<workflow-app name="Sqoop" xmlns="uri:oozie:workflow:0.4"> 
    <start to="Sqoop"/> 
    <action name="Sqoop"> 
     <sqoop xmlns="uri:oozie:sqoop-action:0.2"> 
     <job-tracker>yarn.resourcemanager.address:8040</job-tracker> 
      <name-node>hdfs://hdfs-server:54310</name-node> 
      <command>job --list</command> 
     </sqoop> 
     <ok to="end"/> 
     <error to="kill"/> 
    </action> 
    <kill name="kill"> 
     <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message> 
    </kill> 
    <end name="end"/> 
</workflow-app>

我把這個工作流進我的HDFS。我已經做了Java代碼開始我的工作：

OozieClient wc = new OozieClient("http://localhost:11000/oozie"); 

     Properties conf = wc.createConfiguration(); 
     conf.setProperty(OozieClient.APP_PATH, "hdfs://hdfs_server:54310/hive/testSqoop/sqoop-workflow.xml"); 
     conf.setProperty("queueName", "default"); 

     try { 
      String jobId = wc.run(conf); 
      System.out.println("Workflow job submitted"); 

      while (wc.getJobInfo(jobId).getStatus() == WorkflowJob.Status.RUNNING) { 
       System.out.println("Workflow job running ..."); 
       System.out.println("..." + wc.getJobInfo(jobId).getStatus().toString()); 
       Thread.sleep(10 * 1000); 
      } 
      System.out.println("Workflow job completed ..."); 
      System.out.println(wc.getJobInfo(jobId)); 
     } catch (Exception r) { 
      r.printStackTrace(); 
     }

在Oozie的webinterface我可以看到我的工作運行

2013-05-28 12:42:30,004 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] Start action [[email protected]:start:] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:30,008 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action status=DONE 
2013-05-28 12:42:30,009 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]:start:] [***[email protected]:start:***]Action updated in DB! 
2013-05-28 12:42:30,192 INFO ActionStartXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] Start action [[email protected]] with user-retry state : userRetryCount [0], userRetryMax [0], userRetryInterval [10] 
2013-05-28 12:42:31,389 WARN SqoopActionExecutor:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] credentials is null for the action 
2013-05-28 12:42:42,942 INFO SqoopActionExecutor:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] checking action, external ID [job_1369126414383_0003] status [RUNNING] 
2013-05-28 12:42:42,945 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action status=RUNNING 
2013-05-28 12:42:42,946 WARN ActionStartXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[[email protected]] [***[email protected]***]Action updated in DB! 
2013-05-28 12:47:43,034 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] STARTED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W 
2013-05-28 12:47:43,328 WARN CoordActionUpdateXCommand:542 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] E1100: Command precondition does not hold before execution, [, coord action is null], Error Code: E1100 
2013-05-28 12:47:43,328 INFO KillXCommand:539 - USER[anthonyc] GROUP[-] TOKEN[] APP[Sqoop] JOB[0000000-130528124140043-oozie-anth-W] ACTION[-] ENDED WorkflowKillXCommand for jobId=0000000-130528124140043-oozie-anth-W

當我檢查紗webinterface，我可以看到我的工作，但與狀態失敗

Application application_1369126414383_0003 failed 1 times due to AM Container for appattempt_1369126414383_0003_000001 exited with exitCode: 1 due to: .Failing this attempt.. Failing the application.

我真的不知道什麼是錯的。我需要你的建議。

謝謝〜

來源

2013-05-29 AnthonyC

在Web界面中找到失敗的任務嘗試日誌，您將得到任務失敗的異常。 – SNeumann

你必須檢查作業日誌：

$ oozie job -log <coord_job_id>

明白髮生了什麼。

來源

2016-09-13 19:31:45 ozw1z5rd

Oozie和sqoop配置問題

回答

相關問題