2017-09-12 78 views
1

正在使用Spring Boot 1.5.4.RELEASE Microservice使用ElasticSearch提供的低級Rest Client連接到ElasticSearch 5.5.0實例。如何在使用彈性搜索5.5.0獲得最佳性能時正確關閉原始RestClient?

的pom.xml

<parent> 
    <groupId>org.springframework.boot</groupId> 
    <artifactId>spring-boot-starter-parent</artifactId> 
    <version>1.5.4.RELEASE</version> 
</parent> 

<dependencies> 
    <!-- Spring --> 
    <dependency> 
     <groupId>org.springframework.boot</groupId> 
     <artifactId>spring-boot-starter-web</artifactId> 
    </dependency> 

    <dependency> 
     <groupId>org.springframework.boot</groupId> 
     <artifactId>spring-boot-starter-test</artifactId> 
     <scope>test</scope> 
    </dependency> 

    <!-- Elasticsearch --> 
    <dependency> 
     <groupId>org.elasticsearch</groupId> 
     <artifactId>elasticsearch</artifactId> 
     <version>5.5.0</version> 
    </dependency> 

    <dependency> 
     <groupId>org.elasticsearch.client</groupId> 
     <artifactId>transport</artifactId> 
     <version>5.5.0</version> 
    </dependency> 

    <!-- Apache Commons --> 
    <dependency> 
     <groupId>org.apache.commons</groupId> 
     <artifactId>commons-lang3</artifactId> 
     <version>3.6</version> 
    </dependency> 

    <!-- Jackson --> 
    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-core</artifactId> 
     <version>2.8.9</version> 
    </dependency> 

    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-databind</artifactId> 
     <version>2.8.9</version> 
    </dependency> 

    <dependency> 
     <groupId>com.fasterxml.jackson.core</groupId> 
     <artifactId>jackson-annotations</artifactId> 
     <version>2.8.9</version> 
    </dependency> 

    <!-- Log4j --> 
    <dependency> 
     <groupId>log4j</groupId> 
     <artifactId>log4j</artifactId> 
     <version>1.2.17</version> 
    </dependency> 

    <!-- JUnit --> 
    <dependency> 
     <groupId>junit</groupId> 
     <artifactId>junit</artifactId> 
     <version>4.11</version> 
     <scope>test</scope> 
    </dependency> 

    <!-- Swagger --> 
    <dependency> 
     <groupId>io.springfox</groupId> 
     <artifactId>springfox-swagger2</artifactId> 
     <version>2.6.1</version> 
     <scope>compile</scope> 
    </dependency> 

    <dependency> 
     <groupId>io.springfox</groupId> 
     <artifactId>springfox-swagger-ui</artifactId> 
     <version>2.6.1</version> 
     <scope>compile</scope> 
    </dependency> 
</dependencies> 

一切都設置正確,但一幫命中後,客戶端應用程序報告了一個HTTP 500錯誤,這是什麼出現在日誌文件:

java.io.IOException: Too many open files 
     at sun.nio.ch.IOUtil.makePipe(Native Method) ~[na:1.8.0_141] 
     at sun.nio.ch.EPollSelectorImpl.<init>(EPollSelectorImpl.java:65) ~[na:1.8.0_141] 
     at sun.nio.ch.EPollSelectorProvider.openSelector(EPollSelectorProvider.java:36) ~[na:1.8.0_141] 
     at java.nio.channels.Selector.open(Selector.java:227) ~[na:1.8.0_141] 
     at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.<init>(AbstractMultiworkerIOReactor.java:142) ~[httpcore-nio-4.4.5.jar!/:4.4.5] 
     at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.<init>(DefaultConnectingIOReactor.java:79) ~[httpcore-nio-4.4.5.jar!/:4.4.5] 
     at org.apache.http.impl.nio.client.IOReactorUtils.create(IOReactorUtils.java:43) ~[httpasyncclient-4.1.3.jar!/:4.1.3] 
     at org.apache.http.impl.nio.client.HttpAsyncClientBuilder.build(HttpAsyncClientBuilder.java:666) ~[httpasyncclient-4.1.3.jar!/:4.1.3] 
     at org.elasticsearch.client.RestClientBuilder.createHttpClient(RestClientBuilder.java:202) ~[rest-5.5.0.jar!/:5.5.0] 
     at org.elasticsearch.client.RestClientBuilder.build(RestClientBuilder.java:180) ~[rest-5.5.0.jar!/:5.5.0] 
     at com.myapp.controller.SearchController.getSearchQueryResults(SearchController.java:94) ~[classes!/:1.0] 

Inside SearchController(//註釋後面的第二行是94行):

@RestController 
@RequestMapping("/api/v1") 
public class SearchController { 

    @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json") 
    public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException { 

     // Setup HTTP Headers 
     HttpHeaders headers = new HttpHeaders(); 
     headers.add("Content-Type", "application/json"); 

     // Setup RestClient 
     RestClient restClient = RestClient.builder(new HttpHost("localhost", 9200)) 
     .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() { 
      @Override 
      public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) { 
       return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000); 
      } 
     }).setMaxRetryTimeoutMillis(60000).build(); 

     // Setup query and send and return ResponseEntity... 

    } 
} 

它的顯然,它從未被調用restClient.performRequest()方法關閉後...

所以,我要把它放到我的代碼:

Response response = null; 
try { 
    // Submit Query and Obtain Response 
    response = restClient.performRequest("POST", endPoint, Collections.singletonMap("pretty", "true"), entity); 
} 
catch (IOException e) { 
    LOG.error("\n\n\tException: " + e + "\n\n"); 
    e.printStackTrace(); 
} 
finally { 
    restClient.close(); 
} 

閱讀來自彈性搜索的文檔,該RESTClient實現類是線程-safe ...

另外,請閱讀有關restClient.performRequestAsync()方法的內容,但對線程有點不熟悉,並且文檔中的說明含糊不清。

問題(S):

  1. 是我的解決方案來處理,並關閉了一堆插座資源的最佳方式?

  2. 如果有人能夠向我展示一種更好的方式來使用帶Elastic Search的低級別RestClient,意味着它不會導致同樣的問題,導致HTTP 500不會釋放套接字資源。我應該使用restClient.performRequestAsync?有人可以提供一個例子嗎?

感謝您在百忙之中閱讀本文時...

+0

我覺得restClient.close()在finally塊應該做的伎倆。它不工作嗎? –

+0

你能否確認你沒有使用Elasticsearch的spring引導啓動器? – Val

+0

剩下的客戶端應該只在配置bean中創建一次,作爲依賴注入到控制器中,就是這樣。您不需要爲每個請求創建一個新實例。 – Val

回答

3

這是不是一個很好的做法,對每一個請求創建一個RestClient

@Configuration 
public class ElasticsearchConfig { 

    @Value("${elasticsearch.host}") 
    private String host; 

    @Value("${elasticsearch.port}") 
    private int port; 

    @Bean 
    public RestClient restClient() { 
     return RestClient.builder(new HttpHost(host, port)) 
     .setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() { 
      @Override 
      public RequestConfig.Builder customizeRequestConfig(RequestConfig.Builder requestConfigBuilder) { 
       return requestConfigBuilder.setConnectTimeout(5000).setSocketTimeout(60000); 
      } 
     }).setMaxRetryTimeoutMillis(60000).build(); 
    } 
} 

,然後在SearchController類,你可以注入像這樣(並且還添加了清理方法,當你的容器去關閉restClient比如:你應該通過配置Bean類似下面創建一個實例下):

@RestController 
@RequestMapping("/api/v1") 
public class SearchController { 

    @Autowired 
    private RestClient restClient; 

    @RequestMapping(value = "/search", method = RequestMethod.GET, produces="application/json") 
    public ResponseEntity<Object> getSearchQueryResults(@RequestParam(value = "criteria") String criteria) throws IOException { 

     // Setup HTTP Headers 
     HttpHeaders headers = new HttpHeaders(); 
     headers.add("Content-Type", "application/json"); 

     // Setup query and send and return ResponseEntity... 

     Response response = this.restClient.performRequest(...); 

    } 

    @PreDestroy 
    public void cleanup() { 
     try { 
      logger.info("Closing the ES REST client"); 
      this.restClient.close(); 
     } catch (IOException ioe) { 
      logger.error("Problem occurred when closing the ES REST client", ioe); 
     } 
    } 

}  
+0

謝謝你這麼精心設計的方法!我仍然必須使用restClient.close();在finally {}裏面還是cleanup()方法可以做到這一點?我之前得到錯誤的原因是因爲我沒有調用restClient.close()。我應該在performRequest()之後包含finally並保留cleanup()方法嗎? –

+0

不,您可以保持其他客戶端始終處於打開狀態,只能在清理方法中關閉它。嘗試一下,你會看到;-) – Val

+0

它再次發生,這次使用您的解決方案!似乎RestClient從未關閉過它的連接。我在這裏創建了一個新的帖子,我的發現:https://stackoverflow.com/questions/46559512/elasticsearch-rest-client-still-giving-ioexception-too-many-open-files –