我上MAPR 5.0上運行的Spark 2.1上MAPR 5.0
我得到以下異常,而在本地模式啓動星火
我的火花默認值(重要的配置)
spark.sql.hive.metastore.version 0.13.1
spark.sql.hive.metastore.jars
/opt/mapr/lib/maprfs-5.0.0-mapr.jar:
/opt/mapr/hadoop/hadoop-0.20.2/conf:
/opt/mapr/hadoop/hadoop-0.20.2/lib/protobuf-java-2.5.0.jar:
/opt/hadoopgpl/lib/hadoop-lzo.jar:
/opt/mapr/hadoop/hadoop-2.7.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.0-mapr-1506.jar:
/opt/mapr/hadoop/hadoop-0.20.2/lib/commons-logging-1.1.1.jar:
/opt/mapr/hadoop/hadoop-0.20.2/lib/hadoop-auth-2.7.0-mapr-1506.jar:
/opt/mapr/lib/libprotodefs-5.0.0-mapr.jar:
/opt/mapr/lib/baseutils-5.0.0-mapr.jar:
/opt/mapr/hadoop/hadoop-0.20.2/lib/guava-13.0.1.jar:
/opt/mapr/hadoop/hadoop-0.20.2/lib/hadoop-common-2.7.0-mapr-1506.jar:
/opt/mapr/hadoop/hadoop-0.20.2/lib/commons-configuration-1.6.jar
spark.sql.hive.metastore.sharedPrefixes com.mysql.jdbc,org.postgresql,com.microsoft.sqlserver,oracle.jdbc,com.mapr.fs.shim.LibraryLoader,com.mapr.security.JNISecurity,com.mapr.fs.jni,com.mapr.fs.shim
java.lang.LinkageError: loader (instance of org/apache/spark/sql/hive/client/IsolatedClientLoader$$anon$1): attempted duplicate class definition for name: "com/mapr/fs/jni/MapRConstants"
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:763)
... 104 more
java.lang.IllegalArgumentException: Error while instantiating
'org.apache.spark.sql.hive.HiveSessionState':
星火2.1請幫助我
當時它最終解決了嗎?你可以更新線索與你的言論? – boger