由於OutOfMemory錯誤,我們的solr會不時崩潰。我們在4.0.0版本上仍然存在問題,但在解決以下問題後計劃遷移到最新版本。內存不足(自動完成)內存不足
當我看着Tomcat的日誌我看到下列錯誤:
SEVERE: null:java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space
at org.apache.solr.servlet.SolrDispatchFilter.sendError(SolrDispatchFilter.java:469)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:297)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.lucene.search.FieldComparator$TermOrdValComparator.<init>(FieldComparator.java:1124)
at org.apache.lucene.search.SortField.getComparator(SortField.java:425)
at org.apache.lucene.search.FieldValueHitQueue$MultiComparatorsFieldValueHitQueue.<init>(FieldValueHitQueue.java:110)
at org.apache.lucene.search.FieldValueHitQueue.create(FieldValueHitQueue.java:173)
at org.apache.lucene.search.TopFieldCollector.create(TopFieldCollector.java:1123)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:552)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:507)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:484)
at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:309)
at si.amebis.termania.solr.ExternalSearch.search(ExternalSearch.java:307)
at si.amebis.termania.solr.ExternalSearch.handleRequestBody(ExternalSearch.java:235)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1699)
at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
... 12 more
剛過了自動完成字段的請求(建議您鍵入)。請求詳情如下:
q - *:*
start - 0
rows - 0
fq - (Type:1 OR Type:2)
facet - true
facet.limit - 20
facet.mincount - 1
facet.sort - true
facet.prefix - "mi"
facet.field - "Autocomplete"
--
which returns 8105170 hits
其中自動填充字段被定義爲:
<field name="Autocomplete" type="grams" indexed="true" stored="false" omitNorms="true" required="False" multiValued="true" />
<fieldtype name="grams" class="solr.TextField" positionIncrementGap="100">
<analyzer type="index">
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.ShingleFilterFactory" maxShingleSize="10" outputUnigrams="true" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
</analyzer>
<analyzer type="query">
<tokenizer class="solr.StandardTokenizerFactory" />
<filter class="solr.LowerCaseFilterFactory" />
<filter class="solr.TrimFilterFactory" />
</analyzer>
</fieldtype>
指數細節:
Num document: 4338603
Index size: 10.1 Gb
Ram: 64Gb (-Xmx45000M)
Terms count in Autocomplete field: 70.459.723
我假設上的文本字段等許多方面刻面需要大量的的記憶。
如何計算需要多少內存?是否有更有效的方式提供自動完成(使用短語 - 正克)?
在此先感謝!