Spark上的Hive無法正常工作 - 創建Spark客戶端失敗

Question

執行hive查詢時出現火花引擎錯誤。

Error: 
    Failed to execute spark task, with exception org.apache.hadoop.hive.ql.metadata.HiveException(Failed to create spark client.)' 
    FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.spark.SparkTask 

Hive Console: 
    hive> set hive.execution.engine=spark; 
    hive> set spark.master=spark://INBBRDSSVM15.example.com:7077; 
    hive> set spark.executor.memory=2g; 

    Hadoop - 2.7.0 
    Hive - 1.2.1 
    Spark - 1.6.1 

Answer 1

紗線容器內存比火花執行人 要求小。我設置了YARN容器內存，最大值爲 ，大於Spark Executor Memory + Overhead。檢查 'yarn.scheduler.maximum-allocation-mb'和/或 'yarn.nodemanager.resource.memory-mb'。

Please see Source here