2015-10-23 49 views
2

我想讀30-60 MB在Talend Excel文件,但它給下面的錯誤Java堆錯誤在Talend

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space 
at java.lang.AbstractStringBuilder.<init>(Unknown Source) 
at java.lang.StringBuffer.<init>(Unknown Source) 
at org.apache.xmlbeans.impl.store.Locale$ScrubBuffer.<init>(Locale.java:1804) 
at org.apache.xmlbeans.impl.store.Locale.getScrubBuffer(Locale.java:1904) 
at org.apache.xmlbeans.impl.store.Xobj.getValueAsString(Xobj.java:1205) 
at org.apache.xmlbeans.impl.store.Xobj.fetch_text(Xobj.java:1796) 
at org.apache.xmlbeans.impl.values.XmlObjectBase.get_wscanon_text(XmlObjectBase.java:1332) 
at org.apache.xmlbeans.impl.values.XmlObjectBase.check_dated(XmlObjectBase.java:1269) 
at org.apache.xmlbeans.impl.values.JavaLongHolder.longValue(JavaLongHolder.java:53) 
at org.apache.xmlbeans.impl.values.XmlObjectBase.getLongValue(XmlObjectBase.java:1502) 
at org.openxmlformats.schemas.spreadsheetml.x2006.main.impl.CTRowImpl.getR(Unknown Source) 
at org.apache.poi.xssf.usermodel.XSSFRow.getRowNum(XSSFRow.java:322) 
at org.apache.poi.xssf.usermodel.XSSFSheet.initRows(XSSFSheet.java:229) 
[statistics] disconnected 
[statistics] disconnected 
[statistics] disconnected 
[statistics] disconnected 
[statistics] disconnected 
at org.apache.poi.xssf.usermodel.XSSFSheet.read(XSSFSheet.java:193) 
at org.apache.poi.xssf.usermodel.XSSFSheet.onDocumentRead(XSSFSheet.java:180) 
at org.apache.poi.xssf.usermodel.XSSFWorkbook.onDocumentRead(XSSFWorkbook.java:300) 
at org.apache.poi.POIXMLDocument.load(POIXMLDocument.java:159) 
at org.apache.poi.xssf.usermodel.XSSFWorkbook.<init>(XSSFWorkbook.java:221) 
at de.cimt.talendcomp.tfileexcelpoi.SpreadsheetFile.initializeWorkbook(SpreadsheetFile.java:281) 
at cdif_validation.validation_partsourcingtab3_0_1.Validation_PartSourcingTab3.tFileExcelWorkbookOpen_1Process(Validation_PartSourcingTab3.java:774) 
at cdif_validation.validation_partsourcingtab3_0_1.Validation_PartSourcingTab3.tFileList_1Process(Validation_PartSourcingTab3.java:696) 
at cdif_validation.validation_partsourcingtab3_0_1.Validation_PartSourcingTab3.runJobInTOS(Validation_PartSourcingTab3.java:8268) 
at cdif_validation.validation_partsourcingtab3_0_1.Validation_PartSourcingTab3.runJob(Validation_PartSourcingTab3.java:8110) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.tRunJob_3Process(CDIF_Validation.java:1175) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.tForeach_1Process(CDIF_Validation.java:983) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.tFileList_1Process(CDIF_Validation.java:820) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.tJava_1Process(CDIF_Validation.java:634) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.tPrejob_1Process(CDIF_Validation.java:529) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.runJobInTOS(CDIF_Validation.java:1885) 
at cdif_validation.cdif_validation_0_1.CDIF_Validation.main(CDIF_Validation.java:1722) 

我試圖增加XMS高達1 GB和XMX尺寸高達6 GB。我也嘗試使用磁盤來存儲臨時數據。 如何解決這個問題?

+1

乍一看,擁有1GB的堆空間似乎對於一個60MB的Excel文件來說是完全足夠的。但是,我懷疑Excel文件的內存大小可能遠大於磁盤上的大小。 –

+0

考慮檢查你的代碼,看起來像你有某種內存泄漏。 –

回答

1

與Apache POI有同樣的問題。因爲POI具有較大的內存佔用。現在你可以嘗試做一些工作,比如用較小的零件來處理你的文件,或者試着看看this並做一些「優化」。

另請參閱Apache POI FAQ到第14點以獲取一些建議。

0

默認的talend組件使用比所需內存更多的內存。 我相信這是因爲60MB的xlsx文件是完全未壓縮的內存。即使你只是從1個標籤中讀取1行。

要克服此限制,您可以嘗試添加更多或更多的內存或尋找不同的方法。我通常使用Jan Lolling的tFileExcel組件。 你可以從talend marketplace獲得這些。