FileService用大文件耗盡內存

在開發服務器上，我試圖使用FileService api將數據寫入單個文件。
即使我寫入少量數據，關閉文件（使用close()）也不會釋放內存。我需要使用FileService的closeFinally()來釋放內存。因此，我的猜測是，整個文件被緩存在內存中，直到closeFinally()被調用，如果我想在一個blob中寫入一個巨大的文件，會導致一些問題。
有什麼辦法解決這個內存問題？FileService用大文件耗盡內存

下面是示例代碼：

String veryLongString = "verryyyyllooooonnnnggggssssttttrrrriiiiinnnnnggggg"; 
for (int i = 0; i < 10; ++i) { 
    veryLongString += veryLongString; 
} 
// veryLongString.length() => 51200 char 

FileService FILESERVICE = 
    FileServiceFactory.getFileService(); 

// Create new file 
String newPath = 
    FILESERVICE.createNewBlobFile("application/octet-stream").getFullPath(); 

for (int i = 0; i < 10; ++i) { 
    System.out.println(i); 

    // Open file for append 
    AppEngineFile file = new AppEngineFile(newPath); 
    FileWriteChannel writeChannel = FILESERVICE.openWriteChannel(file, true); 
    OutputStream out = Channels.newOutputStream(writeChannel); 

    // Append some data to this file 
    for (int j = 0; j < 5000; ++j) { 
     out.write(veryLongString.getBytes()); 
    } 
    writeChannel.close(); 
    // Writed 5000*51200 => 256Mo char (to memory, should be 
    // also persisted to disk) 
} 

// Writed all data we want, we finally close the file 
AppEngineFile file = new AppEngineFile(newPath); 
FileWriteChannel writeChannel = FILESERVICE.openWriteChannel(file, true); 
writeChannel.closeFinally();

輸出：

0 
1 
2 
3 
com.google.apphosting.api.ApiProxy$UnknownException: An error occurred for the API request file.Append(). 
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.callInternal(ApiProxyLocalImpl.java:518) 
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(ApiProxyLocalImpl.java:452) 
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.call(ApiProxyLocalImpl.java:430) 
    at java.util.concurrent.Executors$PrivilegedCallable$1.run(Executors.java:463) 
    at java.security.AccessController.doPrivileged(Native Method) 
    at java.util.concurrent.Executors$PrivilegedCallable.call(Executors.java:460) 
    at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) 
    at java.util.concurrent.FutureTask.run(FutureTask.java:138) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) 
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) 
    at java.lang.Thread.run(Thread.java:662) 
Caused by: java.lang.OutOfMemoryError: Java heap space 
    at java.util.Arrays.copyOf(Arrays.java:2786) 
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94) 
    at java.io.OutputStream.write(OutputStream.java:58) 
    at com.google.appengine.api.files.dev.FileMetadata.append(FileMetadata.java:198) 
    at com.google.appengine.api.files.dev.LocalFileService.append(LocalFileService.java:357) 
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) 
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) 
    at java.lang.reflect.Method.invoke(Method.java:597) 
    at com.google.appengine.tools.development.ApiProxyLocalImpl$AsyncApiCall.callInternal(ApiProxyLocalImpl.java:498) 
    ... 10 more

來源

2012-05-09 Fabien

你能張貼一些示例代碼嗎？ –

你在生產中是否也有這個問題？ – proppy

生產中沒有問題。 – Fabien

開發服務器旨在幫助您測試和調試應用程序;可擴展性不是它的主要關注點。你是正確的，它將文件緩存在內存中;這是它應該如何工作的。

來源

2012-05-14 06:49:53

FileService用大文件耗盡內存

回答

相關問題