2017-07-25 59 views
0

我正在運行Jupyter筆記本(Jupyter 1.0.0)與Spark(Spark 2.1.0),並且能夠成功運行Pyspark代碼。但是,當我推出了位於同一個目錄下的兩個筆記本電腦,如下圖所示:

筆記本
| __筆記本1
| __ Notebook2

並推出筆記本1,然後啓動Notebook2,筆記本1的推出和成功的作品,但Notebook2確實由於Spark Context啓動錯誤,無法成功啓動。它看起來像是與spark metastore有關。Metastore錯誤,同時從同一目錄中啓動多個Jupyter Spark筆記本

下面是從星火堆棧跟蹤:

Caused by: ERROR XSDB6: Another instance of Derby may have already booted the database XXXXXXXX/notebooks/metastore_db. 
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) 
at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) 
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.privGetJBMSLockOnDB(Unknown Source) 
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.run(Unknown Source) 
at java.security.AccessController.doPrivileged(Native Method) 
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.getJBMSLockOnDB(Unknown Source) 
at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source) 
at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.store.raw.RawStore$6.run(Unknown Source) 
at java.security.AccessController.doPrivileged(Native Method) 
at org.apache.derby.impl.store.raw.RawStore.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source) 
at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.store.access.RAMAccessManager$5.run(Unknown Source) 
at java.security.AccessController.doPrivileged(Native Method) 
at org.apache.derby.impl.store.access.RAMAccessManager.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.FileMonitor.startModule(Unknown Source) 
at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.db.BasicDatabase$5.run(Unknown Source) 
at java.security.AccessController.doPrivileged(Native Method) 
at org.apache.derby.impl.db.BasicDatabase.bootServiceModule(Unknown Source) 
at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source) 
at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source) 
at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source) 
at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source) 
at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source) 
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source) 
at org.apache.derby.impl.jdbc.EmbedConnection$4.run(Unknown Source) 
at java.security.AccessController.doPrivileged(Native Method) 
at org.apache.derby.impl.jdbc.EmbedConnection.startPersistentService(Unknown Source) 

回答

1

它看起來像你的問題可能與德比,火花默認的數據庫,用於存儲其指令本身。 Derby一次只能由一個實體使用,因此啓動第二個Notebook會使其混淆。我自己遇到了這個問題,並通過將Spark連接到Postgres來解決此問題。

看看這個答案,看看它是否適合你: How to run multiple instances of spark 2.0

相關問題