2017-07-12 85 views
1

我想在我的NiFi jython ExecuteScript處理器中使用Impala的Python DB客戶端。NiFi:無法將外部庫添加到Python的ExecuteScript處理器

我已經安裝了需要的模塊pip install impyla,並在ExecuteScript處理器的Module Directory字段中指定了它的路徑。

之後,我運行腳本,並得到了一個錯誤javax.script.ScriptException: ImportError: No module named _bitarray

隨着pip show bitarray我發現這個模塊的位置,並把它添加到模塊目錄域(用逗號分隔),並重新啓動NiFi以防萬一。

但它沒有幫助。我也有同樣的錯誤。

經過一番研究,我發現bitarray實際上是C擴展。

而且_bitarray出現在bitarray/__init__.py下一行:

from bitarray._bitarray import _bitarray, bitdiff, bits2bytes, _sysinfo 

在其反過來_bitarray存儲裏面_bitarray.so文件在我的模塊目錄中指定的同一目錄中。

有人能幫助我如何加載bitarray模塊?

完整堆棧跟蹤:

2017-07-12 15:00:57,627 ERROR [Timer-Driven Process Thread-5] o.a.nifi.processors.script.ExecuteScript org.apache.nifi.processor.exception.ProcessException: javax.script.ScriptException: ImportError: No module named _bitarray in at line number 1 at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:214) ~[nifi-scripting-processors-1.1.2.jar:1.1.2] at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.2.jar:1.1.2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.2.jar:1.1.2] at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.2.jar:1.1.2] at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.2.jar:1.1.2] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_131] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_131] at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131] at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131] Caused by: javax.script.ScriptException: ImportError: No module named _bitarray in at line number 1 at org.python.jsr223.PyScriptEngine.scriptException(PyScriptEngine.java:202) ~[jython-standalone-2.7.0.jar:na] at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:42) ~[jython-standalone-2.7.0.jar:na] at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31) ~[jython-standalone-2.7.0.jar:na] at javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264) ~[na:1.8.0_131] at org.apache.nifi.processors.script.impl.JythonScriptEngineConfigurator.eval(JythonScriptEngineConfigurator.java:59) ~[nifi-scripting-processors-1.1.2.jar:1.1.2] at org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:204) ~[nifi-scripting-processors-1.1.2.jar:1.1.2] ... 11 common frames omitted Caused by: org.python.core.PyException: null at org.python.core.Py.ImportError(Py.java:328) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_logic(imp.java:912) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_module_level(imp.java:978) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importName(imp.java:1062) ~[jython-standalone-2.7.0.jar:na] at org.python.core.ImportFunction.call(builtin.java:1280) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyObject.call(PyObject.java:431) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.import(builtin.java:1232) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFromAs(imp.java:1156) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFrom(imp.java:1132) ~[jython-standalone-2.7.0.jar:na] at bitarray$py.f$0(/usr/local/lib/python2.7/dist-packages/bitarray/init.py:131) ~[na:na] at bitarray$py.call_function(/usr/local/lib/python2.7/dist-packages/bitarray/init.py) ~[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyCode.call(PyCode.java:18) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromCode(imp.java:436) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:236) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:205) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.loadFromSource(imp.java:651) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.find_module(imp.java:543) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_next(imp.java:840) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_module_level(imp.java:959) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importName(imp.java:1062) ~[jython-standalone-2.7.0.jar:na] at org.python.core.ImportFunction.call(builtin.java:1280) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyObject.call(PyObject.java:431) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.import(builtin.java:1232) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFromAs(imp.java:1156) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFrom(imp.java:1132) ~[jython-standalone-2.7.0.jar:na] at impala.hiveserver2$py.f$0(/usr/local/lib/python2.7/dist-packages/impala/hiveserver2.py:1211) ~[na:na] at impala.hiveserver2$py.call_function(/usr/local/lib/python2.7/dist-packages/impala/hiveserver2.py) ~[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyCode.call(PyCode.java:18) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromCode(imp.java:436) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:236) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:205) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.loadFromSource(imp.java:651) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.find_module(imp.java:543) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyModule.impAttr(PyModule.java:106) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_next(imp.java:842) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_logic(imp.java:904) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_module_level(imp.java:978) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importName(imp.java:1062) ~[jython-standalone-2.7.0.jar:na] at org.python.core.ImportFunction.call(builtin.java:1280) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyObject.call(PyObject.java:431) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.import(builtin.java:1232) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importOneAs(imp.java:1099) ~[jython-standalone-2.7.0.jar:na] at impala.dbapi$py.f$0(/usr/local/lib/python2.7/dist-packages/impala/dbapi.py:193) ~[na:na] at impala.dbapi$py.call_function(/usr/local/lib/python2.7/dist-packages/impala/dbapi.py) ~[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyCode.call(PyCode.java:18) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromCode(imp.java:436) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:236) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.createFromPyClass(imp.java:205) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.loadFromSource(imp.java:651) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.find_module(imp.java:543) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyModule.impAttr(PyModule.java:106) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_next(imp.java:842) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_logic(imp.java:904) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.import_module_level(imp.java:978) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importName(imp.java:1062) ~[jython-standalone-2.7.0.jar:na] at org.python.core.ImportFunction.call(builtin.java:1280) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyObject.call(PyObject.java:431) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.import(builtin.java:1232) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFromAs(imp.java:1156) ~[jython-standalone-2.7.0.jar:na] at org.python.core.imp.importFrom(imp.java:1132) ~[jython-standalone-2.7.0.jar:na] at org.python.pycode._pyx1450.f$0(:5) ~[na:na] at org.python.pycode._pyx1450.call_function() ~[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167) ~[jython-standalone-2.7.0.jar:na] at org.python.core.PyCode.call(PyCode.java:18) ~[jython-standalone-2.7.0.jar:na] at org.python.core.Py.runCode(Py.java:1386) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.eval(builtin.java:497) ~[jython-standalone-2.7.0.jar:na] at org.python.core.builtin.eval(builtin.java:501) ~[jython-standalone-2.7.0.jar:na] at org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259) ~[jython-standalone-2.7.0.jar:na] at org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:40) ~[jython-standalone-2.7.0.jar:na]

回答