storm-deploy提交拓撲結構java.lang.NoClassDefFoundError

Question

在本地集羣中運行拓撲之後，我創建了一個遠程風暴集羣（storm-deploy Nathan）。在使用「包依賴」創建可運行jar之前，我已經從eclipse中的構建路徑中刪除了Storm jars。我的拓撲結構使用storm-kafka-0.9.0-wip16a-scala292.jar，我在構建路徑時將其留在構建路徑中，並在創建可運行jar之前從構建路徑中刪除（僅嘗試解決此問題..）。當我使用下面的命令：

./storm jar /home/ubuntu/Virtual/stormTopologia4.jar org.vicomtech.main.StormTopologia 

它總是回覆：

Exception in thread "main" java.lang.NoClassDefFoundError: OpaqueTridentKafkaSpout 
    at java.lang.Class.getDeclaredMethods0(Native Method) 
    at java.lang.Class.privateGetDeclaredMethods(Class.java:2451) 
    at java.lang.Class.getMethod0(Class.java:2694) 
    at java.lang.Class.getMethod(Class.java:1622) 
    at sun.launcher.LauncherHelper.getMainMethod(LauncherHelper.java:494) 
    at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:486) 
Caused by: java.lang.ClassNotFoundException: OpaqueTridentKafkaSpout 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366) 
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423) 
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) 
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356) 

由於這種拓撲在單個實例上運行良好的AWS作爲運行的JAR，我不能圖什麼i'm在這裏失去了... 是代碼我的主要方法中：

 Config conf = new Config(); 

     OpaqueTridentKafkaSpout tridentSpout = crearSpout(
       kafkadir, "test"); 


     OpaqueTridentKafkaSpout logUpvSpout = crearSpout(kafkadir, 
       "logsUpv"); 

     OpaqueTridentKafkaSpout logSnortSpout = crearSpout(
       kafkadir, "logsSnort"); 

     try { 
      StormSubmitter.submitTopology(
        "hackaton", 
        conf, 
        buildTopology(tridentSpout, logUpvSpout, 
          logSnortSpout)); 
     } catch (AlreadyAliveException | InvalidTopologyException e) { 

      e.printStackTrace(); 
     } 



    } catch (IOException e) { 
     e.printStackTrace(); 
    } catch (TwitterException e) { 
     e.printStackTrace(); 
    } 

} 

private static OpaqueTridentKafkaSpout crearSpout(
     String testKafkaBrokerHost, String topic) { 
    KafkaConfig.ZkHosts hosts = new ZkHosts(testKafkaBrokerHost, "/brokers"); 

    TridentKafkaConfig config = new TridentKafkaConfig(hosts, topic); 
    config.forceStartOffsetTime(-2); 

    config.scheme = new SchemeAsMultiScheme(new StringScheme()); 
    return new OpaqueTridentKafkaSpout(config); 
} 


public static StormTopology buildTopology(OpaqueTridentKafkaSpout tridentSpout, 
     OpaqueTridentKafkaSpout logUpvSpout, 
     OpaqueTridentKafkaSpout logSnortSpout 
     ) throws IOException, 
     TwitterException { 

    TridentTopology topology = new TridentTopology(); 



    topology.newStream("tweets2", tridentSpout) 
      .each(new Fields("str"), new OnlyEnglishSpanish()) 
      .each(new Fields("str"), new WholeTweetToMongo()) 
      .each(new Fields("str"), new TextLangExtracter(), 
        new Fields("text", "lang")).parallelismHint(6) 
      .project(new Fields("text", "lang")) 
      .partitionBy(new Fields("lang")) 
      .each(new Fields("text", "lang"), new Analisis(), 
        new Fields("result")).parallelismHint(6) 
      .each(new Fields("result"), new ResultToMongo()); 


    return topology.build(); 

} 

有沒有什麼辦法可以讓OpaqueTridentKafkaSpout可用？ 預先感謝您

希望it's不是一個愚蠢的cuestion，我非常新手到這個領域

Answer 1

我們可以保持在構建路徑風暴罐子當你生成JAR-具有依賴性，我們只需要告訴行家不捆綁它，像這樣（見「規定」的範圍，這意味着罐子由運行時環境提供的，因此沒有必要捆綁）：但是

<dependency> 
    <groupId>storm</groupId> 
    <artifactId>storm</artifactId> 
    <version>0.9.0-rc2</version> 
    <scope>provided</scope> 
    <exclusions> 
     <exclusion> 
      <groupId>org.slf4j</groupId> 
      <artifactId>slf4j-log4j12</artifactId> 
     </exclusion> 
     <exclusion> 
      <groupId>log4j</groupId> 
      <artifactId>log4j</artifactId> 
     </exclusion> 
    </exclusions> 
</dependency> 

卡夫卡噴出必須包含在jar-with-dependencies中，所以它的maven聲明如下所示：

<dependency> 
     <groupId>storm</groupId> 
     <artifactId>storm-kafka</artifactId> 
     <version>0.9.0-wip16a-scala292</version> 
</dependency> 

爲了驗證東西，您可以隨時解壓縮生成的jar，並手動檢查必要的類是否存在/不存在，因爲它們應該在部署到風暴之前。