Livy錯誤RSCClient：無法連接到上下文

我正在Scala ApI中運行Livy客戶端應用程序，Livy的版本是0.3.0，Spark版本的版本是1.6.0。Livy錯誤RSCClient：無法連接到上下文

以下是主要代碼。

def main(args: Array[String]): Unit = { 
var scalaClient: LivyScalaClient = null 
val livyurl = "http://172.24.4.110:8998" 
try { 
    scalaClient = new LivyClientBuilder(false).setURI(new URI(livyurl)).build().asScalaClient 

// scalaClient.uploadFile(new File("logs/spath2.jar")) 

    val sFuture : ScalaJobHandle[_]= scalaClient.submit(jct => { 
    //Spark_Log() 
    val conf1 = jct.sc.getConf 
    conf1.setAppName("by livy ... ") 
    val sc = jct.sc 
    val sqlContext = jct.sqlctx 
    val dataset: DataFrame = sqlContext.createDataFrame(Seq(
     (1, Vectors.dense(0.0, 0.0, 0.0)), 
     (2, Vectors.dense(0.1, 0.1, 0.1)), 
     (3, Vectors.dense(0.2, 0.2, 0.2)), 
     (4, Vectors.dense(9.0, 9.0, 9.0)), 
     (5, Vectors.dense(9.1, 9.1, 9.1)), 
     (6, Vectors.dense(9.2, 9.2, 9.2)) 
    )).toDF("id", "features") 

    dataset.write.save("usr/root/livy") 
    }) 
} 


finally { 
    if (scalaClient != null) { 
    scalaClient.stop(true) 
    scalaClient = null; 
    } 
}

和李維日誌例外：

17/08/24 05:50:09 INFO InteractiveSession$: Creating LivyClient for sessionId: 30 
 
17/08/24 05:50:09 WARN RSCConf: Your hostname, master-172-24-4-110.neucloud-bd.org, resolves to a loopback address, but we couldn't find any external IP address! 
 
17/08/24 05:50:09 WARN RSCConf: Set livy.rsc.rpc.server.address if you need to bind to another address. 
 
17/08/24 05:50:09 INFO InteractiveSessionManager: Registering new session 30 
 
17/08/24 05:50:10 INFO InteractiveSession: Stopping InteractiveSession 30... 
 
17/08/24 05:50:10 INFO RSCClient: Failing pending job 5efb67ea-1b76-4b80-a7b1-4f2527bf023b due to shutdown. 
 
17/08/24 05:50:10 WARN ContextLauncher: Waiting thread interrupted, killing child process. 
 
17/08/24 05:50:10 INFO InteractiveSession: Stopped InteractiveSession 30. 
 
17/08/24 05:50:10 ERROR RSCClient: Failed to connect to context. 
 
java.util.concurrent.CancellationException 
 
17/08/24 05:50:10 WARN InteractiveSession: (Fail to get rsc uri,java.util.concurrent.ExecutionException: java.util.concurrent.CancellationException) 
 
17/08/24 05:51:09 ERROR SparkYarnApp: Error whiling refreshing YARN state: java.lang.Exception: No YARN application is found with tag livy-session-30-lkjckxwd in 60 seconds. Please check your cluster status, it is may be very busy.

但它是採用Java API做工精細

來源

2017-08-24 Jary zhen

可能是你應該使用 '主機名'，而不是'ip'。

來源

2017-09-04 10:50:06 Anakin

Livy錯誤RSCClient：無法連接到上下文

回答

相關問題