2016-03-05 88 views
0

我正在嘗試將我的R腳本與Storm集成。我的Rbolt代碼是:Storm-R集成

public class RBolt extends ShellBolt implements IRichBolt { 
public RBolt() { 
    super("Rscript", "storm_OR.R"); 
} 
@Override 
public void declareOutputFields(OutputFieldsDeclarer outputFieldsDeclarer) { 
    outputFieldsDeclarer.declare(new Fields("OR")); 
} 

@Override 
public Map<String, Object> getComponentConfiguration() { 
    Config ret = new Config(); 
    ret.setMaxTaskParallelism(1); 
    return ret; 
} 
} 

我收到以下錯誤消息。任何幫助?我確定路徑變量具有R和Rscript的路徑。

17469 [Thread-12-__system] INFO backtype.storm.daemon.executor - Preparing bolt __system:(-1) 
17474 [Thread-12-__system] INFO backtype.storm.daemon.executor - Prepared bolt __system:(-1) 
17480 [Thread-6] INFO backtype.storm.daemon.executor - Loading executor RBolt:[1 1] 
17483 [Thread-6] INFO backtype.storm.daemon.executor - Loaded executor tasks RBolt:[1 1] 
17491 [Thread-6] INFO backtype.storm.daemon.executor - Finished loading executor RBolt:[1 1] 
17491 [Thread-6] INFO backtype.storm.daemon.worker - Launching receive-thread for 8d8a13de-5e87-4e14-b2c2-59b4dfc070c6:1027 
17493 [Thread-14-RBolt] INFO backtype.storm.daemon.executor - Preparing bolt RBolt:(1) 
17496 [Thread-15-worker-receiver-thread-0] INFO backtype.storm.messaging.loader - Starting receive-thread: [stormId: EventProcessing-1-1457335172, port: 1027, thread-id: 0 ] 
17500 [Thread-14-RBolt] INFO backtype.storm.utils.ShellProcess - Storm multilang serializer: backtype.storm.multilang.JsonSerializer 
17510 [Thread-14-RBolt] ERROR backtype.storm.util - Async loop died! 
java.lang.RuntimeException: Error when launching multilang subprocess 

    at backtype.storm.utils.ShellProcess.launch(ShellProcess.java:64) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] 
    at backtype.storm.task.ShellBolt.prepare(ShellBolt.java:99) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] 
    at backtype.storm.daemon.executor$fn__5641$fn__5653.invoke(executor.clj:690) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] 
    at backtype.storm.util$async_loop$fn__457.invoke(util.clj:429) ~[storm-core-0.9.2-incubating.jar:0.9.2-incubating] 
    at clojure.lang.AFn.run(AFn.java:24) [clojure-1.5.1.jar:na] 
    at java.lang.Thread.run(Thread.java:745) [na:1.7.0_67] 
Caused by: java.io.IOException: Cannot run program "Rscript" (in directory "/tmp/933c85f3-f5b5-4a60-b342-7d4969b43d46/supervisor/stormdist/EventProcessing-1-1457335172/resources"): error=2, No such file or directory 

tmp文件夾中的此目錄不存在,並且在運行中創建。任何建議請。

更新:通過在項目的資源文件夾中創建另一個資源文件夾來解決此問題,以便該jar文件包含R文件的資源文件夾。

回答

0

「shell」組件的全部用途是作爲一個獨立進程啓動,因此您的腳本需要執行multilang protocol。 或者你可以找到一個實現協議並具有R集成的庫,如FsStorm:它實現了multilang,你可以通過R type provider調用R函數。