無法在Apache Tez上運行

感謝您在屏幕截圖中詳細發佈此內容。我遵循了您提到的相同步驟，但我無法正確安裝。我遇到的錯誤是無法在Apache Tez上運行

錯誤：無法找到或加載主類org.apache.tez.dag.app.DAGAppMaster

我已經下載了tez的構建版本。 Apache Tez版本：0.8.4，Hadoop版本：2.6.0。

我TEZ-site.xml中是

<?xml version="1.0" encoding="UTF-8"?> 
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> 
    <configuration> 
    <property> 
    <name>tez.lib.uris</name> 
    <value>${fs.defaultFS}/apps/tez-0.8.4,${fs.defaultFS}/apps/tez-0.8.4/lib/</value> 
    </property> 
    </configuration>

和我的.bashrc的配置是：

export HADOOP_HOME=/usr/local/hadoop 
    export HADOOP_INSTALL=$HADOOP_HOME 
    export HADOOP_MAPRED_HOME=$HADOOP_HOME 
    export HADOOP_COMMON_HOME=$HADOOP_HOME 
    export HADOOP_HDFS_HOME=$HADOOP_HOME 
    export YARN_HOME=$HADOOP_HOME 
    export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native 
    export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin 
    export PATH=$PATH:/usr/local/spark/bin 
    export HIVE_HOME=/usr/local/hive 
    export PATH=$PATH:$HIVE_HOME/bin 
    export CLASSPATH=$CLASSPATH:/usr/local/Hadoop/lib/*:. 
    export CLASSPATH=$CLASSPATH:/usr/local/hive/lib/*:. 
    export DERBY_HOME=/usr/local/derby 
    export PATH=$PATH:$DERBY_HOME/bin 
    export CLASSPATH=$CLASSPATH:$DERBY_HOME/lib/derby.jar:$DERBY_HOME/lib/derbytools.jar 
    export HIVE_OPTS="-hiveconf mapreduce.map.memory.mb=4096 -hiveconf mapreduce.reduce.memory.mb=5120" 
    export TEZ_HOME=/usr/local/apache-tez-0.8.4-bin 
    export TEZ_CONF_DIR=$TEZ_HOME/conf 
    export TEZ_JARS=$TEZ_HOME 

    if [ -z "$HIVE_AUX_JARS_PATH" ]; then 
    export HIVE_AUX_JARS_PATH="$TEZ_JARS" 
    else 
    export HIVE_AUX_JARS_PATH="$HIVE_AUX_JARS_PATH:$TEZ_JARS" 
    fi 

    export HADOOP_CLASSPATH=${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/* 
    export CLASSPATH=$CLASSPATH:${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/*:.

我MapReduce的site.xml的是

<configuration> 
     <!--<property> 
      <name>mapreduce.framework.name</name> 
      <value>yarn</value> 
     </property> --> 
     <property> 
     <name>mapreduce.framework.name</name> 
     <value>yarn-tez</value> 
     <description>The runtime framework for executing MapReduce jobs. 
       Can be one of local, classic or yarn. 
     </description> 
     </property> 
    </configuration>

當我嘗試運行示例程序它正在返回跟蹤爲

Failing this attempt. Failing the application. 
    16/07/27 12:52:00 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032 
    16/07/27 12:52:00 INFO client.DAGClientImpl: DAG completed. FinalState=FAILED 
    16/07/27 12:52:00 INFO examples.OrderedWordCount: DAG diagnostics: [Application application_1469604082434_0001 failed 2 times due to AM Container for appattempt_1469604082434_0001_000002 exited with exitCode: 1 
    For more detailed output, check application tracking page:http://AnalyticsLinux.tcs.com:8088/proxy/application_1469604082434_0001/Then, click on links to logs of each attempt. 
    Diagnostics: Exception from container-launch. 
    Container id: container_1469604082434_0001_02_000001 
    Exit code: 1 
    Stack trace: ExitCodeException exitCode=1: 
     at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) 
     at org.apache.hadoop.util.Shell.run(Shell.java:455) 
     at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:715) 
     at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) 
     at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) 
     at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
     at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
     at java.lang.Thread.run(Thread.java:745)

當我看到stderr下的http://localhost:8088時，發現了上面那個。

請幫我解決這個問題。

來源

2016-07-27 Avinash

由於Tez是一個孵化器項目，我們需要下載src並使用maven構建。

Tez 0.8.1 alpha版本的詳細步驟可以參考下面的鏈接。

https://acadgild.com/blog/integrating-apache-tez-with-hadoop/

在建設將在TEZ-ui.Inorder無法不面對這個問題，安裝的git，節點JS和NPM在你的Linux機器，然後開始建設，這將讓你成功構建。

除了提到的步驟，您需要添加的參數在〜/ .bashrc文件以及

export TEZ_CONF_DIR=/usr/local/tez/conf 
export TEZ_JARS=/usr/local/tez/tez 
export HADOOP_CLASSPATH=${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/*:${HADOOP_CLASSPATH}:${JAVA_JDBC_LIBS}:${MAPREDUCE_LIBS} 
export CLASSPATH=$CLASSPATH:${TEZ_CONF_DIR}:${TEZ_JARS}/*:${TEZ_JARS}/lib/*

然後嘗試運行任何MapReduce工作，它成功地工作。

來源

2016-07-28 10:15:38 Avinash

你輸入錯誤的tez包到hdfs中。

HDFS DFS -mkdir /application/tez-0.8.4

焦油-zxvf Apache的TEZ-0.8.4-bin.tar.gz

HDFS DFS -copyFromLocal Apache的TEZ-0.8。 4-bin/share/tez.tar.gz /application/tez-0.8.4

來源

2016-11-17 02:31:33

無法在Apache Tez上運行

回答

相關問題