斯卡拉火花版本不匹配

Question

我在scala中運行字數統計程序時遇到了異常。

Exception in thread "main" java.lang.NoSuchMethodError: scala.Predef$.refArrayOps([Ljava/lang/Object;)Lscala/collection/mutable/ArrayOps; 
    at org.apache.spark.util.Utils$.getCallSite(Utils.scala:1406) 

谷歌搜索的決議我可以理解這種情況發生時，火花和斯卡拉之間不匹配。我PAM dependeny是

<?xml version="1.0" encoding="UTF-8"?> 
<project xmlns="http://maven.apache.org/POM/4.0.0" 
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
    <modelVersion>4.0.0</modelVersion> 

    <groupId>Test</groupId> 
    <artifactId>Test</artifactId> 
    <version>1.0-SNAPSHOT</version> 

    <dependencies> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-core_2.11</artifactId> 
      <version>2.1.0</version> 
     </dependency> 

    </dependencies> 

</project> 

和項目設置階版本斯卡拉-SDK-2.11.8

不知道什麼是錯了這兒過得花費相當多的時間在Maven倉庫中有不同的版本嘗試組合。

從我的本地火花我安裝運行命令scala.util.Properties.versionString

想通了正確的斯卡拉版本我已經選擇了project.but沒有運氣同階的SDK。

非常感謝您的幫助。

Answer 1

下面是一個工作pom.xml來創建超級罐的火花應用程序。通過替換組和工件ID來使用它。運行mvn clean package構建你的超級（單個）jar文件。

<?xml version="1.0" encoding="UTF-8"?> 
<project xmlns="http://maven.apache.org/POM/4.0.0" 
     xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" 
     xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> 
    <modelVersion>4.0.0</modelVersion> 

    <groupId>com.aravind.spark</groupId> 
    <artifactId>wordcount</artifactId> 
    <version>1.0.0-SNAPSHOT</version> 
    <name>${project.artifactId} ${project.version}</name> 

    <properties> 
     <maven.compiler.source>1.8</maven.compiler.source> 
     <maven.compiler.target>1.8</maven.compiler.target> 
     <encoding>UTF-8</encoding> 
     <spark.core.version>2.0.2</spark.core.version> 
     <spark.sql.version>2.0.2</spark.sql.version> 
     <scala.tools.version>2.11</scala.tools.version> 
     <scala.version>2.11.8</scala.version> 
    </properties> 

    <dependencies> 
     <dependency> 
      <groupId>org.scala-lang</groupId> 
      <artifactId>scala-library</artifactId> 
      <version>${scala.version}</version> 
      <scope>provided</scope> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-core_2.11</artifactId> 
      <version>${spark.core.version}</version> 
      <scope>provided</scope> 
     </dependency> 
     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-sql_2.11</artifactId> 
      <version>${spark.sql.version}</version> 
      <scope>provided</scope> 
     </dependency> 
    </dependencies> 

    <build> 
     <sourceDirectory>src/main/scala</sourceDirectory> 
     <testSourceDirectory>src/test/scala</testSourceDirectory> 
     <pluginManagement> 
      <plugins> 
       <plugin> 
        <!-- see http://davidb.github.com/scala-maven-plugin --> 
        <groupId>net.alchim31.maven</groupId> 
        <artifactId>scala-maven-plugin</artifactId> 
        <version>3.2.1</version> 
        <executions> 
         <execution> 
          <goals> 
           <goal>compile</goal> 
           <goal>testCompile</goal> 
          </goals> 
          <configuration> 
           <args> 
            <arg>-dependencyfile</arg> 
            <arg>${project.build.directory}/.scala_dependencies</arg> 
           </args> 
          </configuration> 
         </execution> 
        </executions> 
       </plugin> 
       <plugin> 
        <groupId>org.apache.maven.plugins</groupId> 
        <artifactId>maven-surefire-plugin</artifactId> 
        <version>2.19.1</version> 
        <configuration> 
         <useFile>false</useFile> 
         <disableXmlReport>true</disableXmlReport> 
         <!-- If you have classpath issue like NoDefClassError,... --> 
         <!-- useManifestOnlyJar>false</useManifestOnlyJar --> 
         <includes> 
          <include>**/*Test.*</include> 
          <include>**/*Suite.*</include> 
         </includes> 
        </configuration> 
       </plugin> 
       <plugin> 
        <artifactId>maven-assembly-plugin</artifactId> 
        <version>2.4.1</version> 
        <configuration> 
         <descriptorRefs> 
          <descriptorRef>jar-with-dependencies</descriptorRef> 
         </descriptorRefs> 
        </configuration> 
        <executions> 
         <execution> 
          <id>make-assembly</id> 
          <phase>package</phase> 
          <goals> 
           <goal>single</goal> 
          </goals> 
         </execution> 
        </executions> 
       </plugin> 
       <plugin> 
        <artifactId>maven-compiler-plugin</artifactId> 
        <configuration> 
         <source>${maven.compiler.source}</source> 
         <target>${maven.compiler.target}</target> 
        </configuration> 
       </plugin> 
      </plugins> 
     </pluginManagement> 

     <plugins> 
      <plugin> 
       <groupId>org.apache.maven.plugins</groupId> 
       <artifactId>maven-surefire-plugin</artifactId> 
      </plugin> 
      <plugin> 
       <groupId>net.alchim31.maven</groupId> 
       <artifactId>scala-maven-plugin</artifactId> 
      </plugin> 
      <plugin> 
       <groupId>org.apache.maven.plugins</groupId> 
       <artifactId>maven-shade-plugin</artifactId> 
       <version>2.4.3</version> 
       <executions> 
        <execution> 
         <phase>package</phase> 
         <goals> 
          <goal>shade</goal> 
         </goals> 
         <configuration> 
          <transformers> 
           <!--transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer"> 
            <mainClass>your main job pipeline class</mainClass> 
           </transformer--> 
          </transformers> 
          <createDependencyReducedPom>false</createDependencyReducedPom> 
          <finalName>${project.artifactId}-${project.version}</finalName> 
          <artifactSet> 
           <excludes> 
            <exclude>org.scala-lang:scala-library</exclude> 
            <exclude>org.apache.spark:spark-core_2.10</exclude> 
            <exclude>log4j:log4j</exclude> 
            <exclude>org.slf4j:slf4j-api</exclude> 
            <exclude>ml-apis:xml-apis</exclude> 
            <exclude>org.apache.httpcomponents:httpclient</exclude> 
            <exclude>org.apache.httpcomponents:httpcore</exclude> 
            <exclude>commons-codec:commons-codec</exclude> 
            <exclude>org.apache.ant:ant</exclude> 
            <exclude>org.apache.ant:ant-junit</exclude> 
            <exclude>org.codehaus.jettison</exclude> 
            <exclude>stax:stax-api</exclude> 
            <exclude>commons-configuration:commons-configuration</exclude> 
            <exclude>commons-digester:commons-digester</exclude> 
            <exclude>commons-beanutils:*</exclude> 
            <exclude>com.google.code.findbugs:jsr305</exclude> 
           </excludes> 
          </artifactSet> 
          <filters> 
           <filter> 
            <artifact>*:*</artifact> 
            <excludes> 
             <exclude>META-INF/*.SF</exclude> 
             <exclude>META-INF/*.DSA</exclude> 
             <exclude>META-INF/*.RSA</exclude> 
            </excludes> 
           </filter> 
          </filters> 
         </configuration> 
        </execution> 
       </executions> 
      </plugin> 
     </plugins> 
    </build> 

</project> 

Answer 2

我的整個體驗都在Microsoft堆棧上。我剛開始探索Java技術。如果我在問題中問了一些問題，那麼標記問題的人會通過提供解釋爲什麼它被標記下來幫助我和社區。

任何方式，我已經通過添加參考項目結構>模塊>依賴

screen shot

這解決了problem.Thanks所有誰試圖幫助我。