AWS EMR - 使用Java SDK安裝HUE

Question

我試圖設置一個StepConfig來在我的集羣上安裝並運行HUE。我在下面的方式創建步驟：

private StepConfig newInstallHueStep() { 
    return new StepConfig() 
      .withName("Install Hue") 
      .withActionOnFailure(getDefaultActionOnFailure()) 
      .withHadoopJarStep(stepFactory.newScriptRunnerStep("s3://us-west-1.elasticmapreduce/libs/hue/install-hue")); 
} 

和

private StepConfig newRunHueStep() { 
    return new StepConfig() 
      .withName("Run Hue") 
      .withActionOnFailure(getDefaultActionOnFailure()) 
      .withHadoopJarStep(stepFactory.newScriptRunnerStep("s3://us-west-1.elasticmapreduce/libs/hue/run-hue")); 
} 

結果是步安裝色調是成功的，運行色調失敗。

stderr日誌：

=== Uninstalling hbase 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling security 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling search 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling sqoop 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling zookeeper 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling rdbms 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling spark 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Uninstalling impala 
=== Saved registry at /var/lib/hue/app.reg 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
=== Saved /usr/lib/hue/build/env/lib/python2.6/site-packages/hue.pth 
Running '/usr/lib/hue/build/env/bin/hue syncdb --noinput' with None 
Running '/usr/lib/hue/build/env/bin/hue migrate --merge' with None 
Traceback (most recent call last): 
    File "/usr/lib/hue/build/env/bin/hue", line 9, in <module> 
    load_entry_point('desktop==3.7.1-amzn-2', 'console_scripts', 'hue')() 
    File "/usr/lib/hue/desktop/core/src/desktop/manage_entry.py", line 60, in entry 
    execute_manager(settings) 
    File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 459, in execute_manager 
    utility.execute() 
    File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/__init__.py", line 382, in execute 
    self.fetch_command(subcommand).run_from_argv(self.argv) 
    File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 196, in run_from_argv 
    self.execute(*args, **options.__dict__) 
    File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 232, in execute 
    output = self.handle(*args, **options) 
    File "/usr/lib/hue/build/env/lib/python2.6/site-packages/Django-1.4.5-py2.6.egg/django/core/management/base.py", line 371, in handle 
    return self.handle_noargs(**options) 
    File "/usr/lib/hue/desktop/core/src/desktop/management/commands/install_all_examples.py", line 89, in handle_noargs 
    raise exception 
hadoop.fs.exceptions.WebHdfsException: SecurityException: Failed to obtain user group information: org.apache.hadoop.security.authorize.AuthorizationException: User: hue is not allowed to impersonate hue (error 403) 
Command exiting with ret '255' 

---

我也試着用單步運行色調那麼標準錯誤是：

Command exiting with ret '255' 

Answer 1

的正確方法用Java sdk安裝Hue的方法是發佈安裝色相步驟作爲自舉動作，並將作爲工作流程步驟運行色相。

RunJobFlowRequest runJobFlowRequest = new RunJobFlowRequest("Main", jobFlowInstancesConfig) 
      .withName("SDK_filtering" + System.currentTimeMillis()) 
      .withAmiVersion("3.7.0") 
      .withVisibleToAllUsers(true) 
      .withServiceRole("EMR_DefaultRole") 
      .withJobFlowRole("EMR_EC2_DefaultRole") 
      .withBootstrapActions(
        new BootstrapActionConfig("Install Hue", 
          new ScriptBootstrapActionConfig("s3://us-west-1.elasticmapreduce/libs/hue/install-hue", null))) 
      .withSteps(
        newEnableDebugStep(), 
        newInstallHiveStep(), 
        newInstallPig(), 
        newRunHueStep() 
      );