無法導入火花包

Question

我試圖對Spark JavaKafkaWordCount.java提供的示例Java代碼進行更改，並試圖使用Maven構建它。我正在開發Spark獨立模塊。 我使用下面的pom.xml：

<project> 
    <groupId>com.mycompany.app</groupId> 
    <artifactId>my-app</artifactId> 
    <modelVersion>4.0.0</modelVersion> 
    <name>Simple Project</name> 
    <packaging>jar</packaging> 
    <version>1.0</version> 
    <dependencies> 
     <dependency> <!-- Spark dependency --> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-core_2.10</artifactId> 
      <version>1.5.1</version> 
     </dependency> 

     <dependency> 
      <groupId>junit</groupId> 
      <artifactId>junit</artifactId> 
      <version>4.8.2</version> 
      <scope>test</scope> 
     </dependency> 

     <dependency> 
      <groupId>org.apache.spark</groupId> 
      <artifactId>spark-streaming-kafka_2.10</artifactId> 
      <version>1.5.1</version> 
     </dependency> 
    </dependencies> 
</project> 

當我運行

mvn package 

我收到以下錯誤：

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project my-app: Compilation failure: Compilation failure: 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[17,34] package org.apache.spark.streaming does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[18,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[19,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[20,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[21,43] package org.apache.spark.streaming.api.java does not exist 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,5] cannot find symbol 
[ERROR] symbol: class JavaStreamingContext 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,37] cannot find symbol 
[ERROR] symbol: class JavaStreamingContext 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[39,73] cannot find symbol 
[ERROR] symbol: class Duration 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol 
[ERROR] symbol: class JavaPairReceiverInputDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[48,5] cannot find symbol 
[ERROR] symbol: class JavaPairReceiverInputDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[49,23] cannot access org.apache.spark.streaming.dstream.ReceiverInputDStream 
[ERROR] class file for org.apache.spark.streaming.dstream.ReceiverInputDStream not found 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[51,5] cannot find symbol 
[ERROR] symbol: class JavaDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[58,5] cannot find symbol 
[ERROR] symbol: class JavaDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] /root/my-app/src/main/java/com/mycompany/app/JavaKafkaWordCount.java:[65,5] cannot find symbol 
[ERROR] symbol: class JavaPairDStream 
[ERROR] location: class com.mycompany.app.JavaKafkaWordCount 
[ERROR] -> [Help 1] 
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. 
[ERROR] Re-run Maven using the -X switch to enable full debug logging. 
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles: 
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException 

看來我必須修改我的PATH變量，但我無法找到適當的jar。

Answer 1

你所得到的錯誤信息是：

包org.apache.spark.streaming.api.java不存在

這意味着你缺少以下依賴：

<dependency> 
    <groupId>org.apache.spark</groupId> 
    <artifactId>spark-streaming_2.10</artifactId> 
    <version>1.5.1</version> 
</dependency>