1. 首頁
  2. 問答
  3. 安裝spark時出現很多錯誤

安裝spark時出現很多錯誤

2017-01-08 134 views
-2 
Python 3.5.2 |Anaconda 4.1.1 (x86_64)| (default, Jul 2 2016, 17:52:12) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin 
Type "help", "copyright", "credits" or "license" for more information. 
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties 
Setting default log level to "WARN". 
To adjust logging level use sc.setLogLevel(newLevel). For SparkR, use setLogLevel(newLevel). 
17/01/08 15:27:58 WARN SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0 
17/01/08 15:28:03 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 
17/01/08 15:28:08 WARN SparkContext: Another SparkContext is being constructed (or threw an exception in its constructor). This may indicate an error, since only one SparkContext may be running in this JVM (see SPARK-2243). The other SparkContext was created at: 
org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) 
py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
py4j.Gateway.invoke(Gateway.java:236) 
py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
py4j.GatewayConnection.run(GatewayConnection.java:214) 
java.lang.Thread.run(Thread.java:745) 
17/01/08 15:28:08 WARN SparkContext: Support for Java 7 is deprecated as of Spark 2.0.0 
Traceback (most recent call last): 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/shell.py", line 43, in <module> 
    spark = SparkSession.builder\ 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/sql/session.py", line 169, in getOrCreate 
    sc = SparkContext.getOrCreate(sparkConf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 307, in getOrCreate 
    SparkContext(conf=conf or SparkConf()) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 118, in __init__ 
    conf, jsc, profiler_cls) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 179, in _do_init 
    self._jsc = jsc or self._initialize_context(self._conf._jconf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 246, in _initialize_context 
    return self._jvm.JavaSparkContext(jconf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1401, in __call__ 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value 
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. 
: java.lang.ExceptionInInitializerError 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:397) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) 
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
    at py4j.Gateway.invoke(Gateway.java:236) 
    at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
    at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
    at py4j.GatewayConnection.run(GatewayConnection.java:214) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.net.UnknownHostException: Chenhuis-MacBook-Pro.local: Chenhuis-MacBook-Pro.local: nodename nor servname provided, or not known 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1475) 
    at org.apache.spark.util.Utils$.findLocalInetAddress(Utils.scala:870) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress$lzycompute(Utils.scala:863) 
    at org.apache.spark.util.Utils$.org$apache$spark$util$Utils$$localIpAddress(Utils.scala:863) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:920) 
    at org.apache.spark.util.Utils$$anonfun$localHostName$1.apply(Utils.scala:920) 
    at scala.Option.getOrElse(Option.scala:121) 
    at org.apache.spark.util.Utils$.localHostName(Utils.scala:920) 
    at org.apache.spark.internal.config.package$.<init>(package.scala:189) 
    at org.apache.spark.internal.config.package$.<clinit>(package.scala) 
    ... 13 more 
Caused by: java.net.UnknownHostException: Chenhuis-MacBook-Pro.local: nodename nor servname provided, or not known 
    at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) 
    at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) 
    at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1295) 
    at java.net.InetAddress.getLocalHost(InetAddress.java:1471) 
    ... 22 more 


During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/shell.py", line 47, in <module> 
    spark = SparkSession.builder.getOrCreate() 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/sql/session.py", line 169, in getOrCreate 
    sc = SparkContext.getOrCreate(sparkConf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 307, in getOrCreate 
    SparkContext(conf=conf or SparkConf()) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 118, in __init__ 
    conf, jsc, profiler_cls) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 179, in _do_init 
    self._jsc = jsc or self._initialize_context(self._conf._jconf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/pyspark/context.py", line 246, in _initialize_context 
    return self._jvm.JavaSparkContext(jconf) 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1401, in __call__ 
    File "/Users/ChenhuiChen/Downloads/spark-2.1.0-bin-hadoop2.7/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 319, in get_return_value 
py4j.protocol.Py4JJavaError: An error occurred while calling None.org.apache.spark.api.java.JavaSparkContext. 
: java.lang.NoClassDefFoundError: Could not initialize class org.apache.spark.internal.config.package$ 
    at org.apache.spark.SparkContext.<init>(SparkContext.scala:397) 
    at org.apache.spark.api.java.JavaSparkContext.<init>(JavaSparkContext.scala:58) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) 
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) 
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) 
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526) 
    at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) 
    at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) 
    at py4j.Gateway.invoke(Gateway.java:236) 
    at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) 
    at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) 
    at py4j.GatewayConnection.run(GatewayConnection.java:214) 
    at java.lang.Thread.run(Thread.java:745)

我從網站下載Spark，無法在我的計算機上正確處理它....我沒有做任何事情，只是打開/ bin/pyspark。安裝spark時出現很多錯誤

長時間在線搜索解決方案，無結果。

我需要「構建Spark JAR」嗎？

來源

2017-01-08 Cody

+0

您的計算機上有哪些操作系統？你爲了安裝火花而做了什麼？你執行了哪些操作會導致你附加的錯誤？ – Yaron

+0

它看起來像你在一本macbook上。您是否嘗試了以下步驟： http://genomegeek.blogspot.co.uk/2014/11/how-to-install-apache-spark-on-mac-os-x.html 它也適用於homebrew：_brew install apache-spark_ –

+0

@Yaron感謝您指出如何詳細提問！對於問題有用，我的系統是Sierra。我只是下載spark並安裝它，並沒有做任何其他事情。 – Cody

回答

相關問題

 相關問題