Neo4j非託管擴展和GuardTimeoutException

我非常需要一些關於我在用Java構建的Neo4j非託管擴展中遇到的問題的建議。我創建了一個非常簡單的代碼示例，突出了我的問題。其基本前提是，我想對Neo4j的服務器設置的Neo4j非託管擴展和GuardTimeoutException

org.neo4j.server.webserver.limit.executiontime

的時間爲用戶查詢一個合理的量（可以說2分鐘）它們是通過Cypher，其他端點等進入的。但是我也有一些批處理作業需要通過這個非託管擴展來運行，因此我試圖將它們分解爲幾分鐘的事務處理。我看到的問題是，即使我的每個交易是< 2分鐘，一旦我的流程運行了2分鐘，我就會收到GuardTimeoutException。

下面是示例。在本示例中，我將時間限制爲2000毫秒，所以它不需要我整天進行調試。（越來越可怕接近雖然！）

端點

/** 
* Sample endpoint. 
@GET 
@Path("test") 
@Produces(MediaType.APPLICATION_JSON) 
public Response test(@Context GraphDatabaseService service) { 
    TestFile.test(service); 
    return Response.ok().build(); 
}

邏輯

public static void test(final GraphDatabaseService service) { 
    for (int i = 0; i < 100000; i++) { 
     try (Transaction tx = service.beginTx();) { 
      final Node n = service.createNode(); 
      n.addLabel(testLabel); 
      tx.success(); 
      tx.close(); 
     } 
     System.out.println("Added node"); 
    } 
}

我可以看到，每次交易只需要幾分之一秒，因爲我在大約200個節點加載成功之前時間到。在擊中端點雖然具體的2秒，我得到如下：

org.neo4j.kernel.guard.GuardTimeoutException: timeout occured (overtime=1) 
at org.neo4j.kernel.guard.Guard$Timeout.check(Guard.java:132) 
at org.neo4j.kernel.guard.Guard.check(Guard.java:43) 
at org.neo4j.kernel.InternalAbstractGraphDatabase$5.createNode(InternalAbstractGraphDatabase.java:794) 
at org.neo4j.kernel.impl.api.state.OldTxStateBridgeImpl.nodeCreate(OldTxStateBridgeImpl.java:120) 
at org.neo4j.kernel.impl.api.state.TxStateImpl.nodeDoCreate(TxStateImpl.java:366) 
at org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeCreate(StateHandlingStatementOperations.java:99) 
at org.neo4j.kernel.impl.api.ConstraintEnforcingEntityOperations.nodeCreate(ConstraintEnforcingEntityOperations.java:390) 
at org.neo4j.kernel.impl.api.LockingStatementOperations.nodeCreate(LockingStatementOperations.java:207) 
at org.neo4j.kernel.impl.api.OperationsFacade.nodeCreate(OperationsFacade.java:506) 
at org.neo4j.kernel.InternalAbstractGraphDatabase.createNode(InternalAbstractGraphDatabase.java:1120) 
at **TestFile.test(TestFile.java:15) 
at **test(Jobs.java:48) 
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) 
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 
at java.lang.reflect.Method.invoke(Method.java:606) 
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60) 
at com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205) 
at com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75) 
at org.neo4j.server.rest.transactional.TransactionalRequestDispatcher.dispatch(TransactionalRequestDispatcher.java:139) 
at com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288) 
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) 
at com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108) 
at com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147) 
at com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84) 
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469) 
at com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400) 
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349) 
at com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339) 
at com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416) 
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537) 
at com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699) 
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) 
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:698) 
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1506) 
at org.neo4j.server.guard.GuardingRequestFilter.doFilter(GuardingRequestFilter.java:68) 
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1477) 
at org.neo4j.server.guard.GuardingRequestFilter.doFilter(GuardingRequestFilter.java:68) 
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1477) 
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:503) 
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:211) 
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1096) 
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:432) 
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:175) 
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1030) 
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:136) 
at org.eclipse.jetty.server.handler.HandlerList.handle(HandlerList.java:52) 
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) 
at org.eclipse.jetty.server.Server.handle(Server.java:445) 
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:268) 
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:229) 
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.run(AbstractConnection.java:358) 
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:601) 
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:532) 
at java.lang.Thread.run(Thread.java:744)

正如你可以看到我已經試過tx.success（），tx.close（），任何我能想到的，以使所有的交易完成後就會死亡。任何建議非常感謝！

更新-------------

邁克爾 - 我跟了你建議的步驟 - 我有一個擴展SPIPluginLifecycle一個新的Java類，並還增加了新的配置文件in/src/main/resources/META-INF/services。我可以看到這個文件最終在META-INF/services中的/ plugins中的jar中。但是，我沒有看到這個初始化類被調用。下面是我在啓動時看到的課程和日誌。

public class GraphInitializer implements SPIPluginLifecycle { 
private WebServer webServer; 

@Override 
public Collection<Injectable<?>> start(final GraphDatabaseService graphDatabaseService, final Configuration config) { 
    throw new IllegalAccessError(); 
} 

@Override 
public void stop() { 

} 

@Override 
public Collection<Injectable<?>> start(final NeoServer neoServer) { 
    System.out.println("K starting!"); 
    webServer = getWebServer(neoServer); 
    final Database database = neoServer.getDatabase(); 
    final GraphDatabaseAPI graphDatabaseAPI = database.getGraph(); 
    final Guard guard = graphDatabaseAPI.getDependencyResolver().resolveDependency(Guard.class); 
    final Filter filter = new GuardingRequestFilter(guard, 3000); 
    webServer.addFilter(filter, "/*"); 
    return null; 
} 

private WebServer getWebServer(final NeoServer neoServer) { 
    if (neoServer instanceof AbstractNeoServer) { 
     return ((AbstractNeoServer) neoServer).getWebServer(); 
    } 
    throw new IllegalArgumentException("expected AbstractNeoServer"); 
}

日誌：

2014-10-06 16:14:23.009+0000 INFO [API] Mounted discovery module at [/] 
2014-10-06 16:14:23.014+0000 INFO [API] Mounted REST API at [/db/data/] 
2014-10-06 16:14:23.017+0000 INFO [API] Mounted management API at [/db/manage/] 
2014-10-06 16:14:23.017+0000 INFO [API] Mounted third-party JAX-RS package [***] at [/kristen] 
2014-10-06 16:14:23.017+0000 INFO [API] Mounted webadmin at [/webadmin] 
2014-10-06 16:14:23.017+0000 INFO [API] Mounted Neo4j Browser at [/browser] 
2014-10-06 16:14:23.070+0000 INFO [API] Mounting static content at [/webadmin] from [webadmin-html] 
2014-10-06 16:14:23.124+0000 INFO [API] Mounting static content at [/browser] from [browser] 
12:14:23.126 [main] WARN o.e.j.server.handler.ContextHandler - [email protected]{/,null,null} contextPath ends with/
12:14:23.127 [main] WARN o.e.j.server.handler.ContextHandler - Empty contextPath 
12:14:23.131 [main] INFO org.eclipse.jetty.server.Server - jetty-9.0.5.v20130815 
12:14:23.155 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/,null,AVAILABLE} 
12:14:23.245 [main] INFO o.e.j.w.StandardDescriptorProcessor - NO JSP Support for /webadmin, did not find org.apache.jasper.servlet.JspServlet 
12:14:23.255 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/webadmin,jar:file:/usr/local/Cellar/neo4j/2.1.3/libexec/system/lib/neo4j-server-2.1.3-static-web.jar!/webadmin-html,AVAILABLE} 
12:14:23.668 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/kristen,null,AVAILABLE} 
12:14:23.817 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/db/manage,null,AVAILABLE} 
12:14:24.003 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/db/data,null,AVAILABLE} 
12:14:24.021 [main] INFO o.e.j.w.StandardDescriptorProcessor - NO JSP Support for /browser, did not find org.apache.jasper.servlet.JspServlet 
12:14:24.022 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/browser,jar:file:/usr/local/Cellar/neo4j/2.1.3/libexec/system/lib/neo4j-browser-2.1.3.jar!/browser,AVAILABLE} 
12:14:24.103 [main] INFO o.e.j.server.handler.ContextHandler - Started [email protected]{/,null,AVAILABLE} 
12:14:24.115 [main] INFO o.e.jetty.server.ServerConnector - Started [email protected]{HTTP/1.1}{localhost:7474} 
12:14:24.503 [main] INFO o.e.jetty.server.ServerConnector - Started [email protected]{SSL-HTTP/1.1}{localhost:7473} 
2014-10-06 16:14:24.503+0000 INFO [API] Server started on: http://localhost:7474/ 
2014-10-06 16:14:24.504+0000 INFO [API] Remote interface ready and available at [http://localhost:7474/]

我期待一個新行條目或東西 - 也是我的變化增加超時不實際工作，所以我敢肯定，這些變化沒有采取影響。有什麼額外的我必須添加到neo4j-server.properties，neo4j.properties等？我已經成功添加了設置非託管擴展的行。

謝謝！

來源

2014-10-05 Kristen Marie

小事務使它變慢，因爲每個事務都必須刷新到磁盤。通常在一個tx中加起來多達50k個節點是有意義的，這應該少於一秒。

對於你的防護過濾器，它只是一個servlet過濾器，所以你也可以只檢查Neo4j源代碼的安裝位置，並在你的擴展的init中自己安裝它，但超過你想要批處理的端點。

在：org.neo4j.server.AbstractNeoServer線488

Filter filter = new GuardingRequestFilter(guard, 
      getConfiguration().getInt(Configurator.WEBSERVER_LIMIT_EXECUTION_TIME_PROPERTY_KEY)); 
    webServer.addFilter(filter, "/*");

您可以使用擴展初始化器添加自定義過濾器就像如下所示：

https://github.com/neo4j-contrib/authentication-extension/blob/2.1/src/main/java/org/neo4j/server/extension/auth/AuthenticationExtensionInitializer.java#L81

必須添加一個文件，這樣它包含您的初始化程序類名稱：

https://github.com/neo4j-contrib/authentication-extension/blob/2.1/src/main/resources/META-INF/services/org.neo4j.server.plugins.PluginLifecycle

來源

2014-10-05 10:37:36

謝謝邁克爾，我會給這個配置了一槍，後後續對我的結果！ – 2014-10-06 02:39:53

我遵循了你建議的步驟 - 我有一個新的java類，它擴展了SPIPluginLifecycle，並且還在/ src/main/resources/META-INF/services中添加了新的配置文件。但是，我沒有看到這個初始化類被調用。下面是我在啓動時看到的日誌。 – 2014-10-06 16:17:02

在上面的問題中，查看更多的細節和我的結果！ – 2014-10-06 16:23:32

解決方案發布者Michael Hunger完美地工作，並帶着我自己的一些學習要點。

我不得不在neo4j-server.properties的org.neo4j.server.thirdparty_jaxrs_classes屬性更新更根目錄下，因爲我沒有把我的初始化在同一目錄作爲我的API的Java文件。
start（）方法必須返回一個空集合。
我能夠使用幾種不同的過濾器來爲不同的端點指定時間限制。

下面

最終代碼：

@Override 
public Collection<Injectable<?>> start(final NeoServer neoServer) { 
    webServer = getWebServer(neoServer); 
    final Database database = neoServer.getDatabase(); 
    final GraphDatabaseAPI graphDatabaseAPI = database.getGraph(); 
    final Guard guard = graphDatabaseAPI.getDependencyResolver().resolveDependency(Guard.class); 
    final Filter filter = new GuardingRequestFilter(guard, 600); 
    webServer.addFilter(filter, "/*"); 
    final Filter filter2 = new GuardingRequestFilter(guard, 3000); 
    webServer.addFilter(filter2, "*/batch"); 
    return Collections.emptyList(); 
}

來源

2014-10-06 18:24:16

Neo4j非託管擴展和GuardTimeoutException

回答

相關問題