EHCache3內存不足錯誤

我們希望在我們的springboot微服務中使用EHCache3來緩存二進制對象（PDF和一些元數據）。這些對象的大小在50K到12M之間。使用簡單的測試，我可以加載緩存並查看它的工作情況。EHCache3內存不足錯誤

緩存大型PDF時會發生該問題。下面的堆棧跟蹤顯示

2017-02-03 21:08:47.936 INFO 11 --- [http-nio-8080-exec-4] .e.c.i.r.LoggingRobustResilienceStrategy : Ehcache key 5dd12ae803ea06c4d37afe949090b934 recovered from 

org.ehcache.core.spi.store.StoreAccessException: java.lang.reflect.UndeclaredThrowableException 
    at org.ehcache.core.exceptions.StorePassThroughException.handleRuntimeException(StorePassThroughException.java:72) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1098) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.put(AbstractOffHeapStore.java:306) 
    at org.ehcache.impl.internal.store.tiering.TieredStore.put(TieredStore.java:149) 
    at org.ehcache.core.Ehcache.put(Ehcache.java:196) 
    at com.pany.PdfGeneratorController.convertHtmlToPdf(PdfGeneratorController.java:79) 
    at com.pany.microservice.pdfgenerator.endpoints.PdfGeneratorEndpoint.convertHtmlToPdfUsingPOST(PdfGeneratorEndpoint.java:23) 
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTINNER(PdfgeneratorcontrollerSpringEndpoint.java:100) 
    at com.pany.microservice.pdfgenerator.api.PdfgeneratorcontrollerSpringEndpoint.convertHtmlToPdfUsingPOSTPOST(PdfgeneratorcontrollerSpringEndpoint.java:67) 
    at sun.reflect.GeneratedMethodAccessor137.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:220) 
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:134) 
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:116) 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827) 
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738) 
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85) 
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963) 
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897) 
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970) 
    at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:648) 
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846) 
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:729) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:230) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.web.filter.ApplicationContextHeaderFilter.doFilterInternal(ApplicationContextHeaderFilter.java:55) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.actuate.trace.WebRequestTraceFilter.doFilterInternal(WebRequestTraceFilter.java:105) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:89) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:77) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:107) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:117) 
    at org.springframework.boot.web.support.ErrorPageFilter.access$000(ErrorPageFilter.java:61) 
    at org.springframework.boot.web.support.ErrorPageFilter$1.doFilterInternal(ErrorPageFilter.java:92) 
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) 
    at org.springframework.boot.web.support.ErrorPageFilter.doFilter(ErrorPageFilter.java:110) 
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:192) 
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:165) 
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) 
    at org.apache.catalina.core.StandardContextValve.__invoke(StandardContextValve.java:108) 
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java) 
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472) 
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) 
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) 
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:620) 
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) 
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:349) 
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:783) 
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) 
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:789) 
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455) 
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) 
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) 
    at java.lang.Thread.run(Thread.java:745) 
Caused by: java.lang.reflect.UndeclaredThrowableException: null 
    at com.sun.proxy.$Proxy122.encode(Unknown Source) 
    at org.terracotta.offheapstore.storage.PortabilityBasedStorageEngine.writeMapping(PortabilityBasedStorageEngine.java:64) 
    at org.terracotta.offheapstore.OffHeapHashMap.tryWriteEntry(OffHeapHashMap.java:703) 
    at org.terracotta.offheapstore.OffHeapHashMap.writeEntry(OffHeapHashMap.java:687) 
    at org.terracotta.offheapstore.OffHeapHashMap.computeWithMetadata(OffHeapHashMap.java:1947) 
    at org.terracotta.offheapstore.AbstractLockedOffHeapHashMap.computeWithMetadata(AbstractLockedOffHeapHashMap.java:582) 
    at org.terracotta.offheapstore.concurrent.AbstractConcurrentOffHeapMap.computeWithMetadata(AbstractConcurrentOffHeapMap.java:733) 
    at org.ehcache.impl.internal.store.disk.EhcachePersistentConcurrentOffHeapClockCache.compute(EhcachePersistentConcurrentOffHeapClockCache.java:158) 
    at org.ehcache.impl.internal.store.offheap.AbstractOffHeapStore.computeWithRetry(AbstractOffHeapStore.java:1083) 
    ... 83 common frames omitted 
Caused by: java.lang.reflect.InvocationTargetException: null 
    at sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java) 
    at java.lang.reflect.Method.invoke(Method.java:498) 
    at org.ehcache.impl.internal.store.disk.OffHeapDiskStore$1.invoke(OffHeapDiskStore.java:454) 
    ... 92 common frames omitted 
Caused by: java.lang.OutOfMemoryError: Java heap space

我懷疑問題在於我使用了緩存的配置，但我無法弄清楚。我在堆和磁盤存儲中使用約束值來測試緩存滿時的正確驅逐。通常我會希望新內容從緩存中推出較舊的內容，並降低性能。但是現在我正在擺脫我無法解釋的內存異常。有沒有其他的東西要了解ehcache的分層？

<config 
    xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' 
    xmlns='http://www.ehcache.org/v3' 
    xmlns:jsr107='http://www.ehcache.org/v3/jsr107' 
    xsi:schemaLocation=" 
     http://www.ehcache.org/v3 http://www.ehcache.org/schema/ehcache-core-3.1.xsd 
     http://www.ehcache.org/v3/jsr107 http://www.ehcache.org/schema/ehcache-107-ext-3.1.xsd"> 

    <persistence directory="/root/cachexml"/> 
    <heap-store> 
     <max-object-size unit="MB">20</max-object-size> 
    </heap-store> 

    <cache alias="pdfCache" > 
     <key-type>java.lang.String</key-type> 
     <value-type>com.acquisio.microservice.pdfgenerator.msto.PDFTO</value-type> 
     <expiry> 
      <tti unit="hours">12</tti> 
     </expiry> 

     <resources> 
      <heap unit="entries">4</heap> 
      <disk persistent="true" unit="MB">64</disk> 
     </resources> 
    </cache> 

</config>

來源

2017-02-03 Yan Periard

Ehcache3的堆緩存[不正確支持]（https://groups.google.com/d/msg/ehcache-users/8Mu4LkDOmnc/sYvvgn37CAAJ）具有不同大小的條目，而不是在驅逐之下。這聽起來不像是有計劃解決它。 –

@BenManes這確實看起來像我看到的問題。你知道解決這個問題的方法嗎？ –

Ehcache2是一款非常優秀的產品，我認爲它仍然受到團隊的支持。由於我在[備用緩存]（https://github.com/ben-manes/caffeine）上工作，我對Ehcache3的經驗太有限，無法提供建議。 –

看着stacktrace和Ehcache 3代碼 - 大概就像你沒有給出確切的版本 - 看起來OOME發生在值序列化過程中，所以它可以存儲在磁盤上。

當您在Ehcache 3中的多層緩存中放入一個值時，該值總是寫入最低層 - 磁盤。但是爲了做到這一點，必須將序列化的關鍵和價值。默認情況下，Ehcache 3依賴於Java序列化來完成這項工作，因此需要鍵和值來實現Serializable，根據您的設置情況，PDFTO就是這種情況。

詳細地說，它的工作方式是將對象序列化，然後放入ByteBuffer，然後通過非堆內存將其寫入磁盤。但是這意味着我們需要能夠分配足夠大的空間來繼續。

所以問題不在於堆層，因爲生活在那裏的對象在被傳遞到緩存之前已經被分配了，但是傳輸到了磁盤層。

我建議查看你的整個堆的使用情況，並理解爲什麼你如此受限制，或者檢查你的對象的序列化大小。如果他們有太多外向引用，可能會有潛在的驚喜。

來源

2017-02-07 09:04:28

我確實使用一個小堆重現了您的問題，與您的問題非常相似。你確實需要一些緩衝空間。做一個內存轉儲應該顯示一堆'HeapByteBuffer's漂浮。 – Henri

接受4個條目onheap意味着可以有4個條目。無論大小。所以如果你的堆很小並且你的對象很大，你確實可以得到OOME。

您的解決方案是爲您的堆存儲提供最大內存大小（<heap unit="MB">256</heap>）而不是多個條目。

這樣，在需要時以及在獲得OOME之前（假設最大內存大小低於堆中剩餘的內存），條目將被驅逐。

來源

2017-02-05 20:43:53 Henri

嗨，這不是我所看到的，例如使用64 MB，將填滿堆中的空間，然後拋出內存異常。 - 你也知道爲什麼擁有4項最大大小不會驅逐最古老的項目？我的堆設置爲512米，它應該沒有任何問題。 –

這很有趣，意味着內存泄漏。你使用的是哪個確切版本的ehcache？驅逐是「隨機的」（基於採樣的明智隨機）。因爲在年齡上進行訂購是非常昂貴的，所以在這裏進行折衷。 – Henri

我評論了@ louis-jacomet的回答。由於我所解釋的不是原因，我很確定他說的是。 – Henri

EHCache3內存不足錯誤

回答

相關問題